[SourceKit] Ignore .swiftsourceinfo files in SourceKit

We only show diagnostics that occur in the primary file, so getting the right source locations for symbols coming from loaded modules has no benefit and a slight performance cost.
Also disable it in swift-ide-test by default to match SourceKit's behavior, with an option to enable it.

@swift-ci please test

@swift-ci please test

hmm. Did we have particular performance concern for using .swiftsourceinfo file other than code completion? The way I think of .swiftsourceinfo is that it's an implementation detail of the compiler. It's should be transparent to most clients of the AST, including SourceKitd, unless it's really necessary. Also, not only diagnostics are using Decl::getLoc(), other sourcekitd requests could also benefit from the now valid location, right?

If SourceKit encounters an ambiguous overload, and the overloaded declarations are in a different module, I would expect that the source info files would allow us to refer to the source locations properly in that other module. Did that not work? It could have been one of the nice side benefits of this work.

Build failed
Swift Test OS X Platform
Git Sha - b42f442e20a0ffcb00364c1ed2857f0f6c8caafd

Build failed
Swift Test Linux Platform
Git Sha - b42f442e20a0ffcb00364c1ed2857f0f6c8caafd

@DougGregor do you mean for live issues? Diagnostics (including notes) with a location outside the current buffer aren't shown at the moment. But yeah, if they were shown alongside their associated error/warning they would help with navigating to them. We could do the same with jump-to-definition too (i.e. use the location we now get for references to decls in another module coming back from a CursorInfo request to jump directly to it without using the indexer). The problem with both of these at the moment is that those locations may be stale compared to using the indexer - the index data is updated whenever you edit a file, wile the swiftsourceinfo is only updated when you build, so we actually want to avoid using its location data in favor of the USR + index for the moment until we have some mechanism to trigger it being updated after the corresponding source files are edited.
That said though Xi, Argyrios and I spoke about this in person yesterday and decided to not disable it by default and do so via a flag/setting as it might be helpful for other SourceKit clients, e.g. SourceKit-LSP which I believe doesn't have background indexing yet (so the index data is equally stale), and other open source tools that use SourceKit as a more general mechanism to get semantic info about Swift source rather than supporting an IDE.


Test renderer change

Replaced react-test-renderer with enzyme rendering. React test renderer was not working with some componentes (like patternfly switch).
Using the enzymes shallow and full-dom rendering will us more control over snapshot testing (you can choose how 'deep' you want to render components).
No actual changes were made in tests. Only snapshots and method of generating them were modified

@miq-bot assign @karelhala

@karelhala obviously you should manually check each snapshot...

Overall looks good, however I see some empty commits, can you take a look at the commit hierarchy please?

@karelhala no idea how those empty commits got it. Should be ok now.

@miq-bot remove_label unmergeable


Update to ESMA_cmake v3.0.5

This update is to support MAPL 2.2 reorganization.
Seems zero-diff in my testing, but @sdrabenh give it a shot? I have a lot of builds going and want to make sure I didn't get confused.

@mathomp4 will test shortly

Whoops. Needs to be v3.0.6!


PDB Doesn't correct apply calling convention for fastcall

Version and Platform (required):

Binary Ninja Version: 4.1 Stable

Bug Description:
Binary Ninja can properly figure out the difference between stdcall and fastcall in x86 windows binaries. However when debug info is applied at least in this particular case the calling convention is not correctly set.
Steps To Reproduce:
Please provide all steps required to reproduce the behavior:

Open the shcore.dll binary shared internally with debug info enabled
Open the same binary again without debug info
View 10024bd0  int32_t __stdcall InlineIsEqualGUID(struct _GUID const& arg1, struct _GUID const& arg2) on both
Observe the calling convention is correct before debug info is applied and incorrect afterward

So the pdb loader is accurately reflecting what the pdb says, namely ?InlineIsEqualGUID@@YGHABU_GUID@@0@Z which demangles to int __stdcall InlineIsEqualGUID(struct _GUID const &,struct _GUID const &). Not sure this is our bug...


Constraints as Bounded Expressions

Build "Expressions" from variables that allow arithmetic operations, and add a range to bound them.

this will be interesting to perform in Rust. The three ways I see it are:

Macros that you parse to create a SCIP expression
Operator overloading on variables to build up expressions
String substitution, since you can pass SCIP a string containing the expression, the only work on the wrapper side is then to replace the expression of the variable accepted by russcip with that accepted by SCIP


Allow Conditional Dropping

Hi all,
react-beautiful-dnd allows conditional dropping based on the prop type. This could be easily added to react-kanban as well, such that a type prop can be added to each card and each column. Only Draggables of the same type as the Droppable will work, if the type does not match, then the drop will be cancelled.
this can be used e.g. for:

have a board with "todo", "in progress", "done".
a card can only be moved one column, e.g. from todo to in progess, but not directly to done.

Reference to react-beautiful-dnd: https://github.com/atlassian/react-beautiful-dnd/blob/master/docs/api/droppable.md#conditionally-dropping
I'm willing to try to implement this, if you want, but I think I'm not familiar enough with react-beautiful-dnd to get it done quickly.
Sidenote: In the npm repository, this package is already minified. This is not useful imho, as this will happen anyways when any app is built. Because of this, I always go through the Github repo instead of accessing the files in my IDE.

I implemented this locally, it only needs the new prop type={columnType} in file /Board/components/Column/index.js on line 45.
           <DroppableColumn droppableId={String(children.id)} type={columnType}>
When a card is moved, it can only be dropped in a column with the same type as the column it originated in.

https://github.com/mkmanuel/react-kanban/tree/allow-cond-dropping
I've yet to implement tests etc., but for controlled boards this simple implementation works perfectly

Hi, @mkmanuel. Sorry for the delay.
I can't see how the API is going to work. I'll wait for the specs to look into your implementation.
It's important that your PR closes #351 and #289, those are our needs for today.

Sidenote: In the npm repository, this package is already minified. This is not useful imho, as this will happen anyways when any app is built. Because of this, I always go through the Github repo instead of accessing the files in my IDE.

Is this true for other package bundlers besides webpack? If so, you have got a point.


docs: pattern for sorry pages

This adds a small CSS part and a template for error pages.

@oliverschuerch I restructured the page a little to make it easier to navigate. Can you have another look please?


Editing splat created from lumalabs app

I've been experimenting with integrating splats from the lumalabs app into a threejs experience, and it's amazing! I found a splat editor online and would love to use this to edit my splats. It looks like I need to upload a file, and just provide a link to the capture as in the webgl library. I was wondering if there's a way to get the file from the luma app downloaded so I can edit it before importing it to my threejs app?
Thanks!

I'm going to try to export an .ply file from Luma.ai and import it into Unreal Engine, saw some nice results.

@ericrius1 you've probably figured this out by now, but for anyone else wondering:
Go to https://lumalabs.ai/dashboard/captures and hover over a capture, then click the download arrow. Download the Gaussian Splat under the Unreal Engine section, then unzip the PLY. Don't be fooled, this is a special PLY for the splat and not just a point cloud.
You can edit the splat in various programs from there, though I believe you can only use the semantic mask to "edit" the splats received from the Luma three.js importer:
splats.semanticsMask = LumaSplatsSemantics.FOREGROUND;

@haxiomic @lovelovetrb Is there any way aside from a Luma link to load in other PLY or SPLAT files?

@ericrius1 What online gaussian splat editor did you find/use?
My current use case is making an object scan created via Luma have the object that was scanned be isolated and remove the backdrop & tabletop that Luma's scan included during an object scan (then ideally swapping out the backdrop with something more generic or a skybox of sorts as I only want the object to be included.) Hopefully then able to have embeddable a-la Luma's online splat viewing/embedding.
It would be great if Luma included a way to adjust the bounding box (or similar) to isolate the object that was scanned, but I'm not opposed to doing so manually in the meantime if there's a good set of tools/services I should turn to for this.

While splats.semanticsMask = LumaSplatsSemantics.FOREGROUND; (per https://github.com/lumalabs/luma-web-examples?tab=readme-ov-file#background-removal) is great, I wish there was a quick way to effectively apply what that masking does directly to the splat so it's removed from the splat entirely (as my use case is just an object scan anyway.)
As it is now, it seems like it's still downloading all of the additional background data/assets/etc. while then just hiding it from the render output when the masking is set (causing the splat to take much longer to show than it otherwise would as it still downloads & processes all of that data that's then just not being output.)
Also, the splat still technically has that data available if anyone were to check out the splat data without that masking parameter set when one might want to not have that additional scanned background be accessible to anyone. Hiding via the masking method works okay, but actually removing the background from the splat itself would be best.
Is there anything out there that would make this possible?

Any way to load externaly edited Splat files?
E.g. I crop unnecessary points in https://playcanvas.com/supersplat/editor and want to import edited splat back to Lumalabs Js library?


Rename metadata ForNpgsql methods

Following https://github.com/aspnet/EntityFrameworkCore/issues/16686.
Old methods can remain as obsolete.

Reopening to track additional rename for low-level metadata (https://github.com/aspnet/EntityFrameworkCore/issues/16773).


Parsing errors on various openapi specs

Hi !
I'd like to help out on the project, but I don't really remember how to debug the code anymore, and the logs are not super verbose to explain "where in the spec" it failed
Here are a few examples:
https://bugcrowd.com/openapi/2021-10-28/openapi.yml
Cannot invoke "java.util.Map.entrySet()" because "properties" is null

https://www.secureflag.com/management/api/swagger-config.yaml
attribute is not of type `object`

https://app.swaggerhub.com/apiproxy/schema/file/apis/Veracode/veracode-sca_agent_api_specification/3.0?format=json
attribute paths. '/v3/workspaces/{id}/issues'(get).[status].default is not of type `array`]

I'm mostly interested in using swurg to understand how I could automatically download data from APIs, not really testing them, but this is already showing me how hard it is to parse and use OpenAPI specs... 😅
I'd be curious to see if swurg could have a "fail-safe" mode that just tries the list APIs that don't need any input parameter ? that would make the import more robust ?

Upgraded the project to Burp Montoya and decoupled UI to logic to the best of my knowledge. Now it should be much easier to troubleshot. Any PR would be appreciated! 😀


Rel with string property insertion error

The attached test file triggers assertion
C++ exception with description "Assertion failed in file "/Users/guodong/Developer/kuzu/src/storage/compression/compression.cpp" on line 408: canUpdateInPlace(value, header)" thrown in the test body.

insert_rel_str_1000_1000_1.txt

It's a regression from #2654. The issue is that canUpdateInPlace is skipping nulls, but the null values are still being written as 0, which isn't a value that can be written in-place to an integer bitpacked chunk with a positive offset.
It's being triggered here in StringColumn::write where it's writing a null string as part of the gaps in the CSR data I think, though I'm not sure why the index column in this case started with an offset of 56.
However this shouldn't have any logical effect. It just results in unusual data being stored in null values (specifically, -offset, underflowed and truncated to the bit width used), and since we never attempt to re-compress chunks to make them smaller if the values change, it shouldn't even effect the compression (on the other hand some compression algorithms like delta compression would benefit from knowing where the nulls are so that they can use the value which would be the smallest in that context).
I suppose even passing the null data so that the debug checks can skip validating null values could be helpful though to avoid the error messages.


Be proactive about defending against unauthorized telemetry collection

In the past couple of weeks we've seen plugins in two popular editors taken over by a commercial entity, who has then added seemingly identifiable telemetry collection to those plugins:

Atom (atom-minimap and autocomplete-python): https://github.com/atom-minimap/minimap/issues/588, summary: https://theoutline.com/post/1953/how-a-vc-funded-company-is-undermining-the-open-source-community

Back to autocomplete-python. A number of developers who reviewed the tool found that the changes Kite made were far more invasive than what it had done to Minimap. The new version of autocomplete-python demoted the open source engine it had been using, called Jedi, and enabled Kite’s engine by default.
Kite’s engine required code be processed on Kite’s cloud server, while the previous version did this work on the user’s machine. This is a crucial difference because sending data to the cloud is a privacy issue, especially given the wide-ranging access permission requested by the tool. Developers said they worried that various scenarios could lead to medical data, payment data, and other data that should be kept private being uploaded to Kite. Furthermore, many private companies have policies against uploading data to third parties, which would make Kite unusable for developers at those companies.

Sublime Text (SideBarEnhancements package): https://forum.sublimetext.com/t/rfc-default-package-control-channel-and-package-telemetry/30157, summary: https://www.reddit.com/r/programming/comments/6qwtfz/kite_injected_telemetry_into_the_third_most/dl0psv0/

With the revelation that this has not been limited to one editor, I think it's important that Visual Studio Code implements a proactive policy to either scan uploaded extensions for telemetry collection or sandbox extensions with a permissions system similar to Chrome Extensions / WebExtensions.
I realise that this is a significant undertaking, but with what's come to light in recent weeks, there's a non-zero chance that this is occurring in Visual Studio extensions as well, without user knowledge.

Thanks for bringing this up. This is definitely on our radar and we will investigate how we can protect VS Code extension consumer from such behavior. At this point in time we have no concrete plans yet other than that we are watching this issue very carefully.
Sean - VS Code Team Member

How do you define what telemetry collection is good and what collection is bad? Telemetry is very useful towards extension authors. For me it's not about profit, it's about knowning when errors happen and being able to fix them asap.  If you're using a library like https://github.com/Microsoft/vscode-extension-telemetry it will automatically skip telemetry if the user has it disabled anyways.

"what telemetry collection is good and what collection is bad" anything that can be statistically used to personally identify users, such as geo-location, pieces of what they are writing, their machine data, feature usage, enabled settings, ab-experiments, enabled extensions, etc.

For example, we're seeing a lot of vscode engineers and contributors moving into adtechs and all sorts of weird privacy invasive companies.
@RMacfarlane  contributes to https://github.com/adazzle


GSon Reflection no longer works under Java 17 LTS, use Java 11 JRE for execution.

Will look at fixing this in GSON.

It appears the GSon folks washed their hands of this issue. I will need to write new code with a type adapter so this may be open a while.

Created the new custom reader/writers for GSon. Warning: the JSON may NOT be backward compatible. Start new backups fresh.


feat: Support custom log configuration

Description

Add an input variable to allow us to override the logConfiguration stanza for the ECS task for Atlantis.
Use count instead of for_each for iam policy attachments.

Motivation and Context

Custom log configuration is needed when, for example, running a sidecar container to ship logs to Elasticsearch / Humio / etc.
Fix chicken-and-egg problem when running terraform plan for policies yet to be created (this will recreate policy attachments with indexes instead of keys).

Breaking Changes
resource "aws_iam_role_policy_attachment" "ecs_task_execution" will be recreated due to the nature of using index-based (count) vs. map-name-based (for_each) approach.
How Has This Been Tested?

[ ] I have updated at least one of the examples/* to demonstrate and validate my change(s)
[ ] I have tested and validated these changes using one or more of the provided examples/* projects

[x] I have executed pre-commit run -a on my pull request

The changes have been tested with the latest version of module running in production. The recreated policy attachments had no influence on the running Atlantis instance.
Snippets:
data "aws_iam_policy_document" "ecs_cloudwatch" {
  version = "2012-10-17"
  statement {
    actions = [
      "logs:CreateLogGroup",
    ]
    resources = ["*"]
  }
}

resource "aws_iam_policy" "ecs_cloudwatch" {
  name        = "atlantis-ecs-cloudwatch-policy"
  path        = "/"
  description = "Allows Atlantis to create CloudWatch Log Groups"
  policy      = data.aws_iam_policy_document.ecs_cloudwatch.json
}

module "container_definition_fluentbit" {
  source  = "cloudposse/ecs-container-definition/aws"
  version = "v0.58.1"

  container_name  = "log-router"
  container_image = "amazon/aws-for-fluent-bit:stable"
  essential       = true

  firelens_configuration = {
    type = "fluentbit"

    options = {
      enable-ecs-log-metadata = "true",
      config-file-type        = "file",
      config-file-value       = "/fluent-bit/configs/parse-json.conf",
    }
  }

  log_configuration = {
    logDriver = "awslogs",
    options = {
      awslogs-create-group  = "true",
      awslogs-group         = "atlantis-humio",
      awslogs-region        = var.region,
      awslogs-stream-prefix = "humio-ingester",
    }
  }
}

module "atlantis" {
  source  = "terraform-aws-modules/atlantis/aws"
  version = "3.26.0"
  ...
  # IAM policies
  policies_arn = [
    "arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy",
    ...
    aws_iam_policy.ecs_cloudwatch.arn,
  ]
  ...
  # Extra container definitions
  extra_container_definitions = [module.container_definition_fluentbit.json_map_object]

  custom_log_configuration = {
    logDriver = "awsfirelens",
    options = {
      "Name" : "es",
      "Host" : "cloud.humio.com",
      "Port" : "9200",
      "HTTP_User" : "defaultRepoName",
      "Logstash_Format" : "On",
      "Type" : "flb_type",
      "Time_Key" : "@timestamp",
      "Replace_Dots" : "On",
      "Logstash_Prefix" : "atlantis",
      "Buffer_Size" : "5MB",
      "tls" : "On",
    }
    secretOptions = [
      {
        name      = "HTTP_Passwd",
        valueFrom = data.aws_ssm_parameter.humio_ingest_token.name,
      }
    ]
  }

@antonbabenko @bryantbiggs would you be able to take a look at this?

Kind reminder.

This will be handled with #297

+1 for this change. when can we expect #297 to be implemented?

Remove stale label.

Remove stale label.

This issue has been resolved in version 4.0.0 :tada:


revert to using cjs for lambda

fixes #4933. I don't truly understand why this was failing, but reverting to cjs rather than esm seems to fix it. If it turns out that other apps need esm then we can expose it as a config option
Please don't delete this checklist! Before submitting the PR, please make sure you do the following:

[x] It's really useful if your PR references an issue where it is discussed ahead of time. In many cases, features are absent for a reason. For large changes, please create an RFC: https://github.com/sveltejs/rfcs
[x] This message body should clearly illustrate what problems it solves.
[ ] Ideally, include a test that fails without this PR but passes with it.

Tests

[x] Run the tests with pnpm test and lint the project with pnpm lint and pnpm check

Changesets

[x] If your PR makes a change that should be noted in one or more packages' changelogs, generate a changeset by running pnpm changeset and following the prompts. All changesets should be patch until SvelteKit 1.0

Ah, wait. The switch to esm wasn't a generic modernisation, it was to enable top-level await. I guess we do need the option.
I also wonder if we can skip the esbuild step for lambdas by using @vercel/nft to include the necessary node_modules rather than bundling them into a single file — it's possible that we could then have our cake and eat it.

Vite is considering only supporting ESM output for 3.0. It'd be nice if we could make that work

Vite isn't involved in this part at all

Can we merge this now in favor of unblocking some folk seeing as #4969 might take some time?

yep, i was hoping the nft version would work, but there are some lingering issues to work out


Everoute 1.0.2 agent container crash on kubernetes 1.21.9

root@skydive:~# kubectl logs -n kube-system   everoute-agent-wcwkq everoute-agent -f
time="2022-02-09T09:58:41Z" level=info msg="/var/run/openvswitch/cnibr0.mgmt is disconnected, connecting..."
time="2022-02-09T09:58:41Z" level=info msg="/var/run/openvswitch/cnibr0-policy.mgmt is disconnected, connecting..."
time="2022-02-09T09:58:41Z" level=info msg="Connecting to socket file /var/run/openvswitch/cnibr0.mgmt"
time="2022-02-09T09:58:41Z" level=info msg="/var/run/openvswitch/cnibr0-uplink.mgmt is disconnected, connecting..."
time="2022-02-09T09:58:41Z" level=info msg="Connecting to socket file /var/run/openvswitch/cnibr0-uplink.mgmt"
time="2022-02-09T09:58:41Z" level=info msg="/var/run/openvswitch/cnibr0-cls.mgmt is disconnected, connecting..."
time="2022-02-09T09:58:41Z" level=info msg="Connecting to socket file /var/run/openvswitch/cnibr0-cls.mgmt"
time="2022-02-09T09:58:41Z" level=info msg="New connection.."
time="2022-02-09T09:58:41Z" level=info msg="Connecting to socket file /var/run/openvswitch/cnibr0-policy.mgmt"
time="2022-02-09T09:58:41Z" level=info msg="New connection.."
time="2022-02-09T09:58:41Z" level=info msg="Received Openflow 1.3 Hello message"
time="2022-02-09T09:58:41Z" level=info msg="Received Openflow 1.3 Hello message"
time="2022-02-09T09:58:41Z" level=info msg="New connection.."
time="2022-02-09T09:58:41Z" level=info msg="Received Openflow 1.3 Hello message"
time="2022-02-09T09:58:41Z" level=info msg="Received ofp1.3 Switch feature response: {Header:{Version:4 Type:6 Length:32 Xid:7} DPID:00:00:e6:c4:03:43:30:46 Buffers:0 NumTables:254 AuxilaryId:0 pad:[0 0] Capabilities:79 Actions:0 Ports:[]}"
time="2022-02-09T09:58:41Z" level=info msg="Openflow Connection for new switch: 00:00:e6:c4:03:43:30:46"
time="2022-02-09T09:58:41Z" level=info msg="Switch cnibr0-policy connected"
time="2022-02-09T09:58:41Z" level=info msg="Received ofp1.3 Switch feature response: {Header:{Version:4 Type:6 Length:32 Xid:4} DPID:00:00:42:c8:d1:e0:7b:44 Buffers:0 NumTables:254 AuxilaryId:0 pad:[0 0] Capabilities:79 Actions:0 Ports:[]}"
time="2022-02-09T09:58:41Z" level=info msg="Received ofp1.3 Switch feature response: {Header:{Version:4 Type:6 Length:32 Xid:5} DPID:00:00:72:be:df:c1:64:4a Buffers:0 NumTables:254 AuxilaryId:0 pad:[0 0] Capabilities:79 Actions:0 Ports:[]}"
time="2022-02-09T09:58:41Z" level=info msg="Openflow Connection for new switch: 00:00:72:be:df:c1:64:4a"
time="2022-02-09T09:58:41Z" level=info msg="Switch cnibr0-cls connected"
time="2022-02-09T09:58:41Z" level=info msg="Openflow Connection for new switch: 00:00:42:c8:d1:e0:7b:44"
time="2022-02-09T09:58:41Z" level=info msg="Switch cnibr0-uplink connected"
time="2022-02-09T09:58:41Z" level=info msg="New connection.."
time="2022-02-09T09:58:41Z" level=info msg="Received Openflow 1.3 Hello message"
time="2022-02-09T09:58:41Z" level=info msg="Received ofp1.3 Switch feature response: {Header:{Version:4 Type:6 Length:32 Xid:21} DPID:00:00:4e:f1:42:61:f3:4f Buffers:0 NumTables:254 AuxilaryId:0 pad:[0 0] Capabilities:79 Actions:0 Ports:[]}"
time="2022-02-09T09:58:41Z" level=info msg="Openflow Connection for new switch: 00:00:4e:f1:42:61:f3:4f"
time="2022-02-09T09:58:41Z" level=info msg="Switch cnibr0 connected"
time="2022-02-09T09:58:42Z" level=info msg="Bridge's external-ids are empty"
time="2022-02-09T09:58:42Z" level=info msg="Add file watcher for file: /var/run/openvswitch/db.sock"
panic: runtime error: index out of range [0] with length 0

goroutine 1 [running]:
main.setAgentConf(0xc0000fa090, 0x7f20208b93c0, 0xc000357a10)
        /go/src/everoute/cmd/everoute-agent/config.go:128 +0x911
main.main()
        /go/src/everoute/cmd/everoute-agent/main.go:81 +0x7f5

root@skydive:~# kubectl get po -A
NAMESPACE     NAME                                   READY   STATUS              RESTARTS   AGE
kube-system   coredns-558bd4d5db-j877w               0/1     ContainerCreating   0          16m
kube-system   coredns-558bd4d5db-xhqjr               0/1     ContainerCreating   0          16m
kube-system   etcd-skydive                           1/1     Running             0          17m
kube-system   everoute-agent-wcwkq                   1/2     CrashLoopBackOff    1          27s
kube-system   everoute-controller-867879c886-r7d9f   1/1     Running             0          68s
kube-system   kube-apiserver-skydive                 1/1     Running             0          16m
kube-system   kube-controller-manager-skydive        1/1     Running             0          16m
kube-system   kube-proxy-gzvsx                       1/1     Running             0          16m
kube-system   kube-scheduler-skydive                 1/1     Running             0          16m

This is caused by missing PodCIDR when setup kubernetes cluster, we need more logs here to locate the issue.

Documents: https://github.com/everoute/everoute/blob/main/docs/cni/README.md#kubernetes-cluster


Motion blur shader fails to compile on mac with "vkCreateComputePipelines -1000012000"

Tested versions
v4.3.beta2.official [b75f0485b]
v4.3.beta.custom_build [16f98cd70]
System information
Godot v4.3.beta (16f98cd70) - macOS 14.5.0 - Vulkan (Forward+) - integrated Apple M1 Max - Apple M1 Max (10 Threads)
Issue description
This comes from testing the motion blur implementation by @sphynx-owner. As nobody in the motion blur discord workgroup knows what exactly causes the issue I was advised to report it as is:

Discussion has started here:
https://discord.com/channels/809721752019402762/1250813037221384245/1259859226336034916
Steps to reproduce
Run the motion blur demo from the link below on an apple silicon mac
Minimal reproduction project (MRP)
https://github.com/sphynx-owner/JFA_driven_motion_blur_demo

See https://github.com/godotengine/godot/issues/70772 where the same error code was reproduced.
Does the shader use doubles? This won't work on Metal (and therefore MoltenVK): https://github.com/godotengine/godot/issues/70772#issuecomment-1371379333

Looks like at least one shader uses doubles https://github.com/sphynx-owner/JFA_driven_motion_blur_demo/blob/e3ba524e62bd933e725c49edec305dd72e76260f/MyJumpFloodIteration/jfp_backtracking_experimental.glsl#L84

As stated in the meeting, the solution here is to add a preprocessor for the glsl files to warn users to not use the doubles when they use a Mac.

We'd need to inform users who use doubles on Windows and Linux too, so they're aware that they're losing macOS / iOS compatibility by doing so.
But it needs to be done in a way that users can discard if they decide that their project doesn't aim to support macOS/iOS.

@passivestar If you replace all the double references to float (see diff) and clear the shader cache in the user dir of the project, it fixes this issue.

diff
diff --git a/MyJumpFloodIteration/jfp_backtracking_experimental.glsl b/MyJumpFloodIteration/jfp_backtracking_experimental.glsl
index ad0268d..bb4db19 100644
--- a/MyJumpFloodIteration/jfp_backtracking_experimental.glsl
+++ b/MyJumpFloodIteration/jfp_backtracking_experimental.glsl
@@ -81,9 +81,9 @@ void set_value(bool a, ivec2 uvi, vec4 value, ivec2 render_size)
 float get_motion_difference(vec2 V, vec2 V2, float parallel_sensitivity, float perpendicular_sensitivity)
 {
 	vec2 VO = V - V2;
-	double parallel = abs(dot(VO, V) / max(DBL_MIN, dot(V, V)));
+	float parallel = abs(dot(VO, V) / max(DBL_MIN, dot(V, V)));
 	vec2 perpen_V = vec2(V.y, -V.x);
-	double perpendicular = abs(dot(VO, perpen_V) / max(DBL_MIN, dot(V, V)));
+	float perpendicular = abs(dot(VO, perpen_V) / max(DBL_MIN, dot(V, V)));
 	float difference = float(parallel) * parallel_sensitivity + float(perpendicular) * perpendicular_sensitivity;
 	return clamp(difference, 0, 1);
 }
@@ -103,19 +103,19 @@ vec4 sample_fitness(vec2 uv_offset, vec4 uv_sample)
 //		uv_offset = normalize(sample_velocity) * FLT_MIN;
 //	}

-	double velocity_space_distance = dot(sample_velocity, uv_offset) / max(FLT_MIN, dot(sample_velocity, sample_velocity));
+	float velocity_space_distance = dot(sample_velocity, uv_offset) / max(FLT_MIN, dot(sample_velocity, sample_velocity));

-	double mid_point = params.motion_blur_intensity / 2;
+	float mid_point = params.motion_blur_intensity / 2;

-	double absolute_velocity_space_distance = abs(velocity_space_distance - mid_point);
+	float absolute_velocity_space_distance = abs(velocity_space_distance - mid_point);

-	double within_velocity_range = step(absolute_velocity_space_distance, mid_point);
+	float within_velocity_range = step(absolute_velocity_space_distance, mid_point);

 	vec2 perpen_offset = vec2(uv_offset.y, -uv_offset.x);

-	double side_offset = abs(dot(perpen_offset, sample_velocity)) / max(FLT_MIN, dot(sample_velocity, sample_velocity));
+	float side_offset = abs(dot(perpen_offset, sample_velocity)) / max(FLT_MIN, dot(sample_velocity, sample_velocity));

-	double within_perpen_error_range = step(side_offset, params.perpen_error_thresh * params.motion_blur_intensity);
+	float within_perpen_error_range = step(side_offset, params.perpen_error_thresh * params.motion_blur_intensity);

 	return vec4(absolute_velocity_space_distance, velocity_space_distance, uv_sample.z, within_velocity_range * within_perpen_error_range);
 }

@passivestar If you replace all the double references to float (see diff) and clear the shader cache in the user dir of the project, it fixes this issue

Yes, that's correct. I just confirmed it on discord to the author

Closing as it turned out to be the same as issue as https://github.com/godotengine/godot/issues/70772


feat(cursor): send visual selections to vscode

This PR sends visual selections to vscode as real selections. Discussion from https://github.com/vscode-neovim/vscode-neovim/issues/172.

a fake cursor is created using the extmark api that frees up the primary vscode selection for a proper selection
the Visual highlight is not removed to hide the fact that viw does not trigger an update to vscode
while waiting for https://github.com/neovim/neovim/issues/19708, viw and C-/ will not update vscode visual selection, so immediately doing something like C-S-F (search in files) will not contain the visual selection. However, since the Visual group is still being shown, the issue is minor.
extmarks can now be shown on the last line
VSCodeRange/VSCodeVisual removed

Fix #172, fix #1218, fix #1217, fix #1196, fix #1163, fix #1078, fix #1007, fix #825, fix #507, fix #1155, fix #959, fix #1181, fix #1103, fix #1143

I would appreciate it if anyone could try this out! Recent vsix build: https://github.com/vscode-neovim/vscode-neovim/suites/14098464214/artifacts/788660818

Been working a lot of cursor sync bugs. It's tricky, especially since cursor updates come from many different sources with missing information and different triggers. I somehow have to merge them together, because visual updates have to be treated like cursor updates.
I've actually managed to fix a longstanding bug with this extension, which is that when entering insert mode rapidly, some keys don't get typed in the right pos. Now, everything should be properly executed.
Here's hoping I can get the tests passing soon.

https://github.com/neovim/neovim/issues/11584#issuecomment-568378147
Seems like relying on CursorMoved is not a good solution. I have some other ideas.

neovim/neovim#11584 (comment)
Seems like relying on CursorMoved is not a good solution ...

That issue was closed with the addition of win_viewport, which later got scroll_delta, which was recently improved and is used by https://neovide.dev/ which is quite a nice UI.

I finally got all the tests passing!!
This PR should be ready to go. I'm going to use it at work tomorrow and see how it goes.
Should be ready for another review. Also, this is a good time for anyone to try the PR.

@theol0403 What's the replacement for VSCodeNotifyVisual(a:cmd, a:leaveSelection)?

@wenfangdu you don't need it anymore, just VSCodeNotify

Before I merge, should I mention anywhere (readme, releasenotes, and/or notify when VSCodeNotifyVisual is called) that VSCodeNotifyVisual is removed and should just be replaced with visual selection + VSCodeNotify?

Before I merge, should I mention anywhere (readme, releasenotes, and/or notify when VSCodeNotifyVisual is called) that VSCodeNotifyVisual is removed and should just be replaced with visual selection + VSCodeNotify?

definitely in the release notes, usually best to have a separate # Breaking Changes section

@wenfangdu is it working for you?

@theol0403 One thing I noticed, it takes two Esc to exit both VSCode selection and Neovim visual mode, it supposes only takes one Esc.

@theol0403 One thing I noticed, it takes two Esc to exit both VSCode selection and Neovim visual mode, it supposes only takes one Esc.

Looks like it's caused by my custom Esc keybindings.

Yeah that would only be caused by custom bindings.

@theol0403 Another thing is that when using editor.action.selectAll at the top of the file, it automatically deselects:
https://github.com/vscode-neovim/vscode-neovim/assets/28700378/f4596c2c-a8ed-4210-9b18-e5eaf717f210

@theol0403 Another thing is that when using editor.action.selectAll at the top of the file, it automatically deselects:
https://github.com/vscode-neovim/vscode-neovim/assets/28700378/f4596c2c-a8ed-4210-9b18-e5eaf717f210

@theol0403 Another thing is that when using editor.action.selectAll at the top of a file, it automatically deselects:
https://github.com/vscode-neovim/vscode-neovim/assets/28700378/45b7611d-41c5-4fe5-8cdd-32baf58374b6

Ugh that might take me some work to fix. I didn't pay much attention to the vscode -> nvim selection sync because it already wasn't amazing and its the job of another pr. But this is probably too broken.

@wenfangdu not made yet :)

@benmkramer yeah, that's how it's supposed to work. Not sure why it's not working for you.
Maybe try https://github.com/vscode-neovim/vscode-neovim/suites/14173692178/artifacts/794558736? And you install it by going to extensions pane, menu, install from vsix, and restart.

I went through the following using the download you shared. I also started by uninstalling neovim using the extensions pane to begin with. I also turned off settings sync in case it was doing something weird like reinstalling the main neovim extension automatically 🤷
https://github.com/vscode-neovim/vscode-neovim/assets/2603478/e4db5320-8c72-4b25-b081-68066e18f096

@benmkramer very strange, it looks like it is properly installed, but multiple things aren't working.
For example, your cursor disappears, but it should be replaced with a fake cursor that looks the same.
Do you have empty init.vim/init.lua? What nvim version are you using?

@benmkramer also, out of curiosity, does insert mode work for you?

Do you have empty init.vim/init.lua? What nvim version are you using?

$ nvim --version
NVIM v0.9.1
Build type: Release
LuaJIT 2.1.0-beta3

   system vimrc file: "$VIM/sysinit.vim"
  fall-back for $VIM: "/opt/homebrew/Cellar/neovim/0.9.1/share/nvim"

My init is based on the readme example
if vim.g.vscode then
	require("vscode.init")
else
  -- regular nvim stuff
end

And the vscode init file I have set up is just setting some things around spacing/tabs via vim.opt...
Insert mode, normal mode/traversing text, and visual mode to highlight seem to work OK, but trying to use any kind of vscode action on a visual selection is the main thing I see not working well.
I also tried just now disabling all extensions and ONLY enabled vscode-neovim to the same experience, alongside removing any custom configuration via init.lua for vscode, as well as making sure there are no pending updates for vscode (not running any nightly/insiders build, just the regular public release)

That's very strange. Especially since all tests are passing for macos.
Random note that if you are using packer.nvim, not requiring the plugins is not enough to prevent them from loading, you have to clean as well.
I'll try to think of other causes.

Can you try C-S-F on a selection to see if it is pre-populating the find text field with the selection?

Can you try C-S-F on a selection to see if it is pre-populating the find text field with the selection?

That's very strange. Especially since all tests are passing for macos.
Random note that if you are using packer.nvim, not requiring the plugins is not enough to prevent them from loading, you have to clean as well.
I'll try to think of other causes.

Interesting. I'm using lazy.nvim, but can try getting things to a more pure state!

Can you try C-S-F on a selection to see if it is pre-populating the find text field with the selection?

It is not. Maybe based on your other comment though, some plugins are still getting loaded in and throwing everything off even without the import of my plugins setup. Will try that. 🙏

My hunch is that the custom lua scripts provided by this PR aren't being properly loaded. However, this is confusing me because a) switching to insert mode still works (modechange now originates from those lua scripts) and b) you don't seem to be talking about any errors showing up in the vscode-neovim output panel.
Essentially, the way this PR works is that there are two triggers that update the cursor:

cursor moved
mode changed

If either of those triggers are met, it will update the cursor. Additionally, if we are in visual mode, it will decorate a fake cursor and create a visual selection. It seems like that last action is failing, but I'm not sure why.

If anyone else has a mac, I would appreciate it if you could try to test this PR! Unfortunately I do not have one, and don't have the slightest clue why it isn't working for @benmkramer .

@theol0403 Another thing is that when using editor.action.selectAll at the top of a file, it automatically deselects:
cmd+a.mp4
Other than that, it looks pretty good so far.

@theol0403 @benmkramer My tests are done on an MBP, my nvim version is 0.8.3.

@theol0403 Another thing is that when using editor.action.selectAll at the top of a file, it automatically deselects:
cmd+a.mp4
Other than that, it looks pretty good so far.

@theol0403 @benmkramer My tests were done on an MBP, my nvim version is 0.8.3. I recommend trying without any nvim or vscode configs and extensions except the ones associated with vscode-neovim.

@wenfangdu thanks for confirming, will dig more into potential plugin/config conflicts.
@theol0403 No need to hold up on account of me. Seems like there is good potential it's something with my local configuration. If it is helpful though, happy to keep trying here and sharing updates, or even connect synchronously/screenshare if there is value to understand the issue further.

@benmkramer I'd be happy to troubleshoot further, whatever way you think would be most effective. I want to be sure I don't break the extension on a major release.

@wenfangdu I've added a new commit which changes how vscode selections are sent to nvim. However, it does not yet fix your problem, due to bugs with viewport scrolling. https://github.com/vscode-neovim/vscode-neovim/issues/1261
What it does add is intuitive mouse selection/double click/form new selection.
Do you think breaking C-a in insert mode is a dealbreaker?
@justinmk I realised my issue. What this new commit does is convert a vscode selection into two mouse events sent to nvim, which handles all the selection logic. However, when doing "select all", mouse events must be sent for positions outside of the screen. However, nvim_input_mouse deals with screen positions, not buffer positions, so I cannot tell it to select outside of the viewport.
Do you have any suggestions for this? Is there another API that would let me create a visual selection anywhere in the buffer, or input the mouse for external positions?
Somehow I feel like the best solution would be to scroll neovim along with the mouse events :/

@wenfangdu try out latest commit, everything should be fixed now, selectAll should work as well

Do you think breaking C-a in insert mode is a dealbreaker?

@theol0403 Did you mean selectAll in normal mode? Since selectAll certainly works in insert mode.

@wenfangdu yes of course, I was referring to the bug you found (fixed now anyway)

@theol0403 But it reproduced with the latest commit, selection and selectAll will auto-deselect:
https://github.com/vscode-neovim/vscode-neovim/assets/28700378/ca6a9977-d2fe-4bb4-add9-72a6fddf50ad

@theol0403 But it reproduced with the latest commit, selection and selectAll will auto-deselect:
https://github.com/vscode-neovim/vscode-neovim/assets/28700378/ca6a9977-d2fe-4bb4-add9-72a6fddf50ad

Do you think breaking C-a in insert mode is a dealbreaker?

I've kinda got used to 😂, if it's too time-consuming to fix, you can address it in another PR.

I see the confusion, I meant normal mode, sorry. I'll keep chipping away at it and will merge if I can't figure it out. Seems like the mouse selection is still a little buggy.

@benmkramer I'd be happy to troubleshoot further, whatever way you think would be most effective. I want to be sure I don't break the extension on a major release.

Not sure what happened here, but pulled down these latest changes, rebuilt, reinstalled. With my nvim config and vscode config all as they were (e.g. not disabling everything/removing plugins) this is working now.
Working well now! Not totally sure if I just caught some new commits that solved it, or something in my env changed enough to stop breaking it. Maybe the act of cleaning up all the configs/plugins and then putting them back helped in some way.

@wenfangdu can you try selectAll with the newest commit?

Latest broke column selection, will fix tomorrow.

For those curious, all my commits since the last passing test have been to add two-way sync, now you can intuitively use the mouse or vscode selections and it will produce nvim selections.

@wenfangdu can you try selectAll with the newest commit?

@theol0403 Yes, it's working now. 🎉

@wenfangdu can you try selectAll with the newest commit?

@theol0403 Yes, it's working now. 🎉
A small thing is that according to https://www.conventionalcommits.org/en/v1.0.0/, feat!(cursor) should be written as feat(cursor)!.

Alright, I'm almost done!
This PR totally refactors the cursor manager, and makes it faster, smoother, and more intuitive. Visual selection works well, mouse interaction works well, and a lot of sync bugs are solved.
There is a little bug with the fake cursor on a line with emojis, but the bug is in the highlight manager and not the scope of this PR.
Once I can figure out why the windows tests stopped working, I'll merge!
@justinmk now would be a good time for a final review if you want

I can't be sure, but I think I've really improved the performance of the cursor manager, events are now being debounced properly, and things feel a lot smoother. Also, now the mouse selection is perfect.

Very exciting, thank you for persisting with this.

This is a major improvement. It feels like a new generation of this extension. Thank you so much for all the effort you put in this!


Fix LGTM nits

See https://lgtm.com/projects/g/EricssonResearch/spindump/

Tentatively resolved in commit ef85d47 but need to wait 24 hours to see if the LGTM refresh verifies all issues as corrected.

LGTM says:
Excellent, there aren't any alerts for EricssonResearch/spindump right now!


Add memcached as a state store

Describe the proposal
Implement MongoDB as a state store .

I'm working on it. Will push something soon.

@daft300punk could you please update the description for this.

closed via #71


[Feature/extensions] Initial draft for registering settings dynamically

Description
Initial Draft PR for registering settings dynamically.
Next steps:

Understand how the new registered setting can be replicated across the cluster
See failures in gradle check

Issues Resolved
#3434
Check List

[ ] New functionality includes testing.

[ ] All tests pass

[ ] New functionality has been documented.

[ ] New functionality has javadoc added

[ ] Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

This seems deceptively simple, so something must be wrong ;)

Closing this. Merge conflicts killing this.
I'll send out a fresh PR.


in block in combine_namespace_routes': undefined method reject' for nil:NilClass (NoMethodError)

This issue looks similar to the issue fixed in #508.
~/workspace/ruby/test_grape_swagger hanfei@hanfei-rmbp
0:59:43 ❯ cat Gemfile
source 'https://rubygems.org'

gem 'grape', '~> 0.19'
gem 'grape-swagger', '~> 0.27'

~/workspace/ruby/test_grape_swagger hanfei@hanfei-rmbp
0:59:58 ❯ cat api.rb
require 'grape'
require 'grape-swagger'

module TheApi
  class ParentLessNamespaceApi < Grape::API
    route_param :animal do
      route_param :breed do
        resource :queues do
          route_param :queue_id do
            resource :reservations do
              desc 'Lists all reservations specific type of animal of specific breed in specific queue'
              get do
                { bla: 'Bla Black' }
              end
            end
          end
        end
      end
    end
  end
end

class ParentLessApi < Grape::API
  prefix :api
  mount TheApi::ParentLessNamespaceApi
  add_swagger_documentation version: 'v1'
end

~/workspace/ruby/test_grape_swagger hanfei@hanfei-rmbp
1:00:06 ❯ ruby -v
ruby 2.4.0p0 (2016-12-24 revision 57164) [x86_64-darwin16]

~/workspace/ruby/test_grape_swagger hanfei@hanfei-rmbp
1:00:32 ❯ ruby api.rb
/Users/hanfei/.rvm/gems/ruby-2.4.0/gems/grape-swagger-0.27.2/lib/grape-swagger.rb:100:in `block in combine_namespace_routes': undefined method `reject' for nil:NilClass (NoMethodError)
        from /Users/hanfei/.rvm/gems/ruby-2.4.0/gems/grape-swagger-0.27.2/lib/grape-swagger.rb:95:in `each'
        from /Users/hanfei/.rvm/gems/ruby-2.4.0/gems/grape-swagger-0.27.2/lib/grape-swagger.rb:95:in `combine_namespace_routes'
        from /Users/hanfei/.rvm/gems/ruby-2.4.0/gems/grape-swagger-0.27.2/lib/grape-swagger.rb:49:in `add_swagger_documentation'
        from api.rb:26:in `<class:ParentLessApi>'
        from api.rb:23:in `<main>'

@mur-wtag
The related spec is one of some specs never get executed because the filenames of those specs not having the suffix _spec. And 3 of 4 are failed now.
Here is the list:
spec/swagger_v2/description_not_initialized.rb
spec/swagger_v2/host.rb
spec/swagger_v2/parent_less_namespace.rb
spec/swagger_v2/reference_entity.rb

@qqshfox Yes, you are absolutely right. It's getting:
# ./spec/swagger_v2/parent_less_namespace_spec.rb:26
NoMethodError:
       undefined method `each' for nil:NilClass
     # ./lib/grape-swagger/doc_methods/tag_name_description.rb:7:in `block in build'

this type of error.
Let's fix it. :)


Questions (FAQs?) after a first read

Hi, I took a look over the https://atos.open-control.org/ page. It was super helpful! Here are the main open questions I still have. I think they fall onto the really basic/high-level overview side of things... I actually feel like I have some understanding of the SSP and the mechanics of the process itself.
For context: I came across this guide by way of @afeld ... my program team at Census is confused about exactly when/how we need an ATO for a couple of projects. We hadn't gotten a straight answer on this question, though were recently linked a policy document that described some of the ATO types in Commerce. It helped clarify 3. below because Commerce has something called a "time-bound" ATO for potentially pilot purposes.

At what point in the development of a system do you need an ATO? You can develop/prototype internally without one? Are you relying on some other ATO (or perhaps a platform-level ATO) during that time? Does a system prototype that involves confidential/sensitive data in the prototyping phase (e.g. a machine learning model's specs derived from sensitive data) change these parameters?
I didn't fully understand the "when to start" tip (tip 1 under "General"). Specifically, what is a "security boundary"? The NIST glossary definition (which is currently down for me, so can't link) is hilariously unclear. This confusion may also be due to my not understanding when exactly you need an ATO.
How does the "Impact" of a system change as it scales? For example, at Census we're thinking of initially piloting a new data ingestion tool in a low-stakes way. Availability/Integrity almost don't matter in that case, since site could go down / we could completely lose the data and be fine. As long as we don't cause a data breach. However, when we go to "production" with this (in the sense of a full-scale survey production), obviously those parameters change significantly.
Someone here at Census mentioned a cost in the tens-of-thousands of dollars to go through the ATO process. I'd say that detail on that (unless it's too agency-specific) would be helpful.

Here are some responses to your questions, but hopefully someone can add more based on your specific context:

In general you need to authorize a system when it is going to be relied on to process, store, or interact with federal data. So if you're developing a prototype with non-production data then an authorization isn't required. Before you accept federal data into the system you need an ATO. To start prototyping with sensitive data is generally not a good idea.

A security boundary is also called an authorization boundary or accreditation boundary. It defines the edge of your product and where your responsibility to protect federal data starts/ends. Any system outside of that which you communicate with would be a "System interconnection" and is generally called out in the system security plan (SSP). https://csrc.nist.gov/glossary/term/accreditation-boundary

I don't have a ton of experience in your space with this particular aspect, but I imagine it's a similar idea: You should assess your system at the impact level you calculate it to be when it's fully operational. See https://csrc.nist.gov/glossary/term/impact and specifically FIPS 199. Your impact will be the high water mark of your CIA impact. So if you have low confidentiality, low integrity, and moderate availability impact it will should be tested at a moderate impact level. **Take this all with a grain of salt as this is how it works in cloud service provider world.

No experience on the cost of the ATO process for an agency, hopefully someone else can chime in on 3 and 4. For a CSP it can be way more than 10s of thousands but I imagine it's significantly cheaper for an agency.

Hey Christian, there's a lot of information below. I have to break this down for myself to understand better. Your agency should have training on how the Risk Management Framework (RMF) and the Assessment and Authorization (A&A; the ATO process) is implemented. They're required to.
Remember that the ATO is for legal for regulatory requirements to show that the project meets the minimum security controls.

At what point in the development of a system do you need an ATO?

Determining if an ATO is needed during the management and planning phase of the project. If the system is planning to go to production, it needs to add the various requirements (FIPS 199/200, 800-37) of the assessments to its planning and timeline.

You can develop/prototype internally without one?

If the system is a prototype without plans to go to/be used in production, then you can argue avoiding the requirements of the ATO. Unless it's prototyping something to be used in the future, then it's a tough argument. It's always an argument. Most Information System Security Officers (ISSOs) are risk averse and will tell you to do the ATO. If you can get the business unit to state otherwise, it'll be an argument.

Are you relying on some other ATO (or perhaps a platform-level ATO) during that time?

Generally, all systems rely on other ATOs of some sort. These are common with General Support Systems (GSS) such as systems that manage the enterprise. VMWare infrastructure, Microsoft ecosystem, PKI systems, etc. Now with FedRAMP, application-specific solutions can rely on the CSP's ATO.

Does a system prototype that involves confidential/sensitive data in the prototyping phase (e.g. a machine learning model's specs derived from sensitive data) change these parameters?

To provide emphasis, DOD and members of intelligence prototype all the time. If it's in production -- where the sensitive data is located, then somebody has to accept the risk and that's what the ATO is for. If there's sanitized data in test, it's back to the argument detailed above. Sometimes not needed.

I didn't fully understand the "when to start" tip (tip 1 under "General"). Specifically, what is a "security boundary"? The NIST glossary definition (which is currently down for me, so can't link) is hilariously unclear. This confusion may also be due to my not understanding when exactly you need an ATO.

800-37 details when to begin, and it's back to the start of the SDLC with management and planning. In the past it was "system boundary", now it seems to be "authorization boundary". It's the parts, bits, components of the system to be authorized for operate in production. This doesn't include existing systems already authorized if there's interconnectivity somewhere.

How does the "Impact" of a system change as it scales? For example, at Census we're thinking of initially piloting a new data ingestion tool in a low-stakes way. Availability/Integrity almost don't matter in that case, since site could go down / we could completely lose the data and be fine. As long as we don't cause a data breach. However, when we go to "production" with this (in the sense of a full-scale survey production), obviously those parameters change significantly.

Without knowing much about the project I'd suggest talking to the architecture team(s) and work to determine the impact of the system as it scales. The Configuration Management teams are also required to have this knowledge as part of their role expectations. Hopefully capacity planning is a requirement. Also, take a look at your agency's Privacy Impact Assessment (PIA) documents.

Someone here at Census mentioned a cost in the tens-of-thousands of dollars to go through the ATO process. I'd say that detail on that (unless it's too agency-specific) would be helpful.

Unfortunately this is a true fact for almost everywhere. I've seen agencies budget small cost for Assessment & Authorization and go way over budget, and others budget a flat rate enterprise wide. If an agency has 92 systems, that's $9.2 million USD annually. Some military branches have well over 3,000 information systems.  As a former auditor, assessor, and certifier, I've always referred to Information Assurance as a mandatory sunk cost. It's why the need to include security requirements early in the project are important. The #1 rule of FISMA is cost-effective security.

Closing, but we can re-open if there are further questions.


Question / Help: Scaffold custom dialog (The49.Maui.BottomSheet)

Hi @adospace
This might be out of scope as it relates to a 3rd party library.
I'm currently trying to integrate a BottomSheet with the help of The49.Maui.BottomSheet. To achieve this, I used your input from the documentation via How to deal with custom dialogs/popups?.
The code I've implemented for the scaffolding / wrapping part looks like the following.
Bottom Sheet Scaffold / Wrapper

[Scaffold(typeof(The49.Maui.BottomSheet.BottomSheet))]
public partial class BottomSheet
{
    protected override void OnAddChild(VisualNode widget, MauiControls.BindableObject childNativeControl)
    {
        if (childNativeControl is MauiControls.View content)
        {
            NativeControl.EnsureNotNull();
            NativeControl.Content = content;
        }

        base.OnAddChild(widget, childNativeControl);
    }

    protected override void OnRemoveChild(VisualNode widget, MauiControls.BindableObject childNativeControl)
    {
        NativeControl.EnsureNotNull();

        if (childNativeControl is MauiControls.View content && NativeControl.Content == content)
        {
            NativeControl.Content = null;
        }

        base.OnRemoveChild(widget, childNativeControl);
    }
}

public class BottomSheetHost : Component
{
    private bool _isShown;
    private Action<DismissOrigin>? _onDismissedAction;

    private The49.Maui.BottomSheet.BottomSheet? _bottomSheet;
    private readonly Action<The49.Maui.BottomSheet.BottomSheet?>? _nativePopupCreateAction;

    public BottomSheetHost(Action<The49.Maui.BottomSheet.BottomSheet?>? nativePopupCreateAction = null)
    {
        _nativePopupCreateAction = nativePopupCreateAction;
    }

    public BottomSheetHost IsShown(bool isShown)
    {
        _isShown = isShown;
        return this;
    }

    public BottomSheetHost OnDismissed(Action<DismissOrigin> action)
    {
        _onDismissedAction = action;
        return this;
    }

    protected override void OnMounted()
    {
        InitializeBottomSheet();
        base.OnMounted();
    }

    protected override void OnPropsChanged()
    {
        InitializeBottomSheet();
        base.OnPropsChanged();
    }

    private void InitializeBottomSheet()
    {
        if (!_isShown || MauiControls.Application.Current == null)
        {
            return;
        }

        MauiControls.Application.Current.Dispatcher.DispatchAsync(async () =>
        {
            if (ContainerPage == null || _bottomSheet is null)
            {
                return;
            }

            await _bottomSheet.ShowAsync();
        });
    }

    public override VisualNode Render()
    {
        if (!_isShown)
        {
            return null!;
        }

        var children = Children();
        return new BottomSheet(r =>
            {
                _bottomSheet = r;
                _nativePopupCreateAction?.Invoke(r);
            })
            {
                children[0]
            }
            .Detents([
                new MediumDetent()
            ])
            .SelectedDetent(new MediumDetent())
            .OnDismissed((_, origin) => _onDismissedAction?.Invoke(origin));
    }
}

Additionaly, here are parts of the app code I've used.
App Code

MauiProgram.cs
public static class MauiProgram
{
    public static MauiApp CreateMauiApp()
    {
        var builder = MauiApp.CreateBuilder();
        builder
            .UseMauiReactorApp<MainPage>()
            .UseBottomSheet()
            .ConfigureFonts(fonts =>
            {
                fonts.AddFont("OpenSans-Regular.ttf", "OpenSansRegular");
                fonts.AddFont("OpenSans-Semibold.ttf", "OpenSansSemibold");
            });

        return builder.Build();
    }
}

MainPage.cs
public class MainPage : Component
{
    public override VisualNode Render() => NavigationPage(new BottomSheetPage());
}

BottomSheetPage.cs
public class BottomSheetPage : Component<BottomSheetPageState>
{
    private The49.Maui.BottomSheet.BottomSheet? _bottomSheet;

    public override VisualNode Render()
    {
        return new ContentPage
        {
            new Grid
            {
                new Button("Show Bottom Sheet")
                    .HCenter()
                    .VCenter()
                    .OnClicked(() => ShowBottomSheet()),

                new BottomSheetHost(r => _bottomSheet = r)
                    {
                        new VStack(spacing: 10)
                        {
                            new Label("Hello from Bottom Sheet!"),
                            new Button("Close", async () => await _bottomSheet!.DismissAsync())
                                .HCenter()
                                .VCenter()
                        }
                    }
                    .IsShown(State.IsShown)
                    .OnDismissed(result =>
                    {
                        Console.WriteLine(result);
                        SetState(s => s.IsShown = false);
                    })
            }
        };
    }

    private void ShowBottomSheet()
    {
        if (State.IsShown)
        {
            return;
        }

        SetState(s => s.IsShown = true);
    }
}

I now run into the problem that the .NET MAUI handlers from the BottomSheet implementation are being called during the Render() phase. At this stage certain properties are still null which the handler expects to be non-null. For example in this method the Controller is null because ShowAsync has not yet been called.
public static void MapBackground(BottomSheetHandler handler, BottomSheet view)
{
    // The Controller property should not be null.
    view.Controller.UpdateBackground();
}

This leads to NullReferenceExceptions which can prevent the rendering of the content inside the BottomSheet.
In a classic MVVM application the samples show that a custom BottomSheet is instantiated in a Command and then directly displayed via ShowAsync.
var page = new SimplePage(); // SimplePage inherits from BottomSheet / is a BottomSheet
page.ShowAsync(Window);

This way the mentioned Controller property is never null as the controller is created inside the ShowAsyncmethod which happens before the component is acually rendered to the page (see here).
Is there a way to achieve the same behavior with MAUI Reactor?
Thanks a lot for your help!

Hi,
nothing wrong with your approach that works most of the time but the BottomSheet is a "special" control that works best using an imperative code.
This is why I've created a repo for that (that repo contains implementations for a variety of 3rd party controls):
https://github.com/adospace/mauireactor-integration/blob/main/The49/Pages/BottomSheetManager.cs
I have used BottomSheet in some of my applications. this is one of them:
https://github.com/adospace/mauireactor-samples/tree/main/TaskApp
https://www.youtube.com/watch?v=q-oM2PO0ZtU&ab_channel=AdolfoMarinucci

You're amazing! Thank you very much!


Time Travel: zoom too insensitive in Firefox

In Firefox, to zoom out from a range of 6 minutes to a range of 1 hour, takes 55(!) full drags of my scroll wheel (one drag is about 8 clicks, so about 440 clicks), which is way too insensitive.
On Chrome, it takes just 3 full drags.
For comparison, zooming by the same 10x factor on Google Maps takes just 13 clicks.

Following #2777, the zoom in FF is now not quite sensitive enough, and, moreover, is massively different from Chrome.
To zoom by a factor of 10, e.g. go from fully zoomed in, when the timeline covers about 6 minutes, to a level where it covers 60 minutes, takes 5 clicks on my mouse wheel on FF and 33 on Chrome.
using my trackpad gives the same result - one full "pinch" on FF goes from 6 minutes to 60, and in Chrome to just 12.
I reckon the correct sensitivity is actually somewhere inbetween - the FF zoom feels too coarse, and the Chrome zoom too fine. The 13 clicks that Google Maps takes for 10x zoom is probably about right.


Model weirdness with Entity Model Features

Tested with EMI on Fabric 0.15.7 and Cubical Critters (https://modrinth.com/resourcepack/cubical-critters)

@Partonetrain What minecraft version are you using and what version of the resource pack is used?

1.20.1, resource pack v1.0, Entity Model Features 1.20.1-1.3

I did some testing and the issue only occurs when a resourcepack changes vanilla mob models with .jem files. EMF allows you to load in .jem files from optifine.
Tested with Optifine instead of [EMF] Entity Model Features and [ETF] Entity Texture Features and the issue went away.
That indicates that you should file a bug report on the EMF github. And hopefully you can get more help there.


add formSubmition error handling to ui

If submitting the form fails, it must be error toast explaining what is happened

To implement this, the throwing an errors should be precise (try catch on specific logic block).
Also should avoid nesting try...catch, because rethrowing error seems not optimal.
try {
  throw1
  throw2
  throw3
} catch () // will get 1,2 or 3

try {
  try {
    t1, t2, t3
} catch // will get 1,2 or 3 {
   throw 4
}
} catch // will get 4

https://github.com/skorphil/nextjs-form/assets/6762009/714dc4ff-96d9-4ffb-af73-d83c6dcac3fc


improvement: Introduce wg_tags to handle default_tags on provider level

PR o'clock
closes: #1578, #1455(?), #1248(?)

Description
When default_tags are used on aws provider level, and the same tags are defined as input to this module, we end up with this issue:
default_tags always shows an update
If "tags" var is not set, no tags will be applied to resources which defines tag_specification.
So, either we have default_tags and do not set "tags". which leaves us with "empty" tag_specifications or we don't use default_tags.
This PR introduces new variable "wg_tags", which will be used on resources which define dynamic tags/tag_specifications if "tags" is not set. Backward compatibility is therefore fully retained.
There will be changes only if wg_tags is set, otherwise the module works the same as before.
Checklist

[X] README.md has been updated after any changes to variables and outputs. See https://github.com/terraform-aws-modules/terraform-aws-eks/#doc-generation

@antonbabenko this is actually quite important with the current behaviour of the AWS provider's default tags. we had to disable the usage of default tags in TF state with this module due to not having the ability to set non-default tags only on  tag_specifications and dynamic tag resources

If anyone wants to work on this one, please check this PR (around data source aws_default_tags) - https://github.com/terraform-aws-modules/terraform-aws-autoscaling/pull/169/files#diff-dc46acf24afd63ef8c556b77c126ccc6e578bc87e3aa09a931f33d9bf2532fbbR1

The solution proposed by @antonbabenko is much better. I will close this PR and try to implement the proposed solution when I find some time.


Improvement: Leave hook handler needs passing done callback

Leave hook handler needs passing done callback the same way it's being passed into the enter hook handler. This one is needed to prevent leaving pages containing unsaved form data, etc.

Good suggestion. I'll add it to my list for the rewrite of the library which is happening now.

Navigo has been re-written and in the latest version 8.0.0 the latest hook accepts a function that allow us to block (or completely prevent) the leaving of a route. Documentation here.
P.S.
Version 8.0.0 is not officially released but can be installed via npm install navigo@beta or yarn add navigo@beta. Migration guide is available here https://github.com/krasimir/navigo/blob/big-rewrite/CHANGELOG.md#migration-guide and the new documentation here https://github.com/krasimir/navigo/blob/big-rewrite/DOCUMENTATION.md.


[BUG] Closing or opening browsers that share the same chrome instance affects emulation on other active browsers

System info

Playwright Version: [v1.29.0] (Also tested on 1.35.1 and the bug still exists)
Operating System: [macOS 13.4]
Browser: [Chromium]

Source code
Test file (self-contained)
The test shows the issue where PRINT media type on page2 disappears after we close browser1.
package repro

import com.microsoft.playwright.Browser;
import com.microsoft.playwright.Browser.NewContextOptions;
import com.microsoft.playwright.BrowserContext;
import com.microsoft.playwright.Page;
import com.microsoft.playwright.Playwright;
import com.microsoft.playwright.options.Media;
import java.util.Map;

/**
 * Need to run command below on terminal first
 *

 * "/Applications/Google Chrome.app/Contents/MacOS/Google Chrome" --remote-debugging-port=9222
 *

 */
public class EmulationTest {
    private static final String DEVURL = "http://localhost:9222/";
    // A random webpage that has different css setting when media type is set to PRINT
    private static final String HTML = "https://www.jegsworks.com/lessons/web-2/html/step-mediaprint.htm";

    public static void main(String[] args) throws InterruptedException {
        Playwright playwright1 = Playwright.create(
                new Playwright.CreateOptions().setEnv(Map.of("PLAYWRIGHT_SKIP_BROWSER_DOWNLOAD", "true")));
        Browser browser1 = playwright1.chromium().connectOverCDP(DEVURL);
        BrowserContext context1 = browser1.newContext(new NewContextOptions());
        Page page1 = context1.newPage();
        page1.navigate(HTML);

        // Note that if we emulate media type here (before we create browser2), the emulation will also go away after
        // browser 2 has been created

        Playwright playwright2 = Playwright.create(
                new Playwright.CreateOptions().setEnv(Map.of("PLAYWRIGHT_SKIP_BROWSER_DOWNLOAD", "true")));
        Browser browser2 = playwright2.chromium().connectOverCDP(DEVURL);
        BrowserContext context2 = browser2.newContext(new NewContextOptions());
        Page page2 = context2.newPage();
        page2.navigate(HTML);

        System.out.println("Checking if emulations stick");
        // False
        System.out.println("page1 has print media type before setting media type to PRINT: " + isPrintMediaType(page1));
        page1.emulateMedia(new Page.EmulateMediaOptions().setMedia(Media.PRINT));
        // True
        System.out.println("page1 has print media type after setting media type to PRINT: " + isPrintMediaType(page1));

        // False
        System.out.println("page2 has print media type before setting media type to PRINT: " + isPrintMediaType(page2));
        page2.emulateMedia(new Page.EmulateMediaOptions().setMedia(Media.PRINT));
        // True
        System.out.println("page2 has print media type after setting media type to PRINT: " + isPrintMediaType(page2));

        System.out.println("Closing page1 and see if media type changes for page2");
        page1.close();
        // True
        System.out.println("page2 has print media type after closing page1: " + isPrintMediaType(page2));

        context1.close();
        // True
        System.out.println("page2 has print media type after closing context1: " + isPrintMediaType(page2));

        browser1.close();
        // False --> doing something on browser1 shouldn't affect anything browser2
        System.out.println("page2 has print media type after closing browser1: " + isPrintMediaType(page2));

        playwright1.close();
        // False --> expected since browser1 closed
        System.out.println("page2 has print media type after closing playwright1: " + isPrintMediaType(page2));

        // Clean up
        page2.close();
        context2.close();
        browser2.close();
        playwright2.close();
    }

    private static String isPrintMediaType(Page page) {
        return page.evaluate("() => matchMedia('print').matches").toString();
    }
}

Steps

Run "/Applications/Google Chrome.app/Contents/MacOS/Google Chrome" --remote-debugging-port=9222 on terminal to start a chrome instance
Run the code to see outputs -- all outputs should match the comments

Expected
I would expect each browser to be isolated from each other. Interacting with one browser (opening or closing) should not affect pages on a different browser.
Actual
Opening or closing a new browser messes up media type of other browsers. This also affects dark-mode light-mode if each browser has different configs.

It is actually `page.emulateMedia call that affects the other context when connected over CDP:
package org.example;

import com.microsoft.playwright.Browser;
import com.microsoft.playwright.Browser.NewContextOptions;
import com.microsoft.playwright.BrowserContext;
import com.microsoft.playwright.Page;
import com.microsoft.playwright.Playwright;
import com.microsoft.playwright.options.Media;
import java.util.Map;

/**
 * Need to run command below on terminal first
 *

 * "/Applications/Google Chrome.app/Contents/MacOS/Google Chrome" --remote-debugging-port=9222
 *

 */
public class EmulationTest {
  private static final String DEVURL = "http://localhost:9222/";
  // A random webpage that has different css setting when media type is set to PRINT
  private static final String HTML = "https://www.jegsworks.com/lessons/web-2/html/step-mediaprint.htm";

  public static void main(String[] args) throws InterruptedException {
    Playwright playwright1 = Playwright.create(
      new Playwright.CreateOptions().setEnv(Map.of("PLAYWRIGHT_SKIP_BROWSER_DOWNLOAD", "true")));
    Browser browser1 = playwright1.chromium().connectOverCDP(DEVURL);
    BrowserContext context1 = browser1.newContext(new NewContextOptions());
    Page page1 = context1.newPage();
    page1.navigate(HTML);

    page1.emulateMedia(new Page.EmulateMediaOptions().setMedia(Media.PRINT));
    // True
    System.out.println("page1 has print media type after setting media type to PRINT: " + isPrintMediaType(page1));

    // Note that if we emulate media type here (before we create browser2), the emulation will also go away after
    // browser 2 has been created

    Playwright playwright2 = Playwright.create(
      new Playwright.CreateOptions().setEnv(Map.of("PLAYWRIGHT_SKIP_BROWSER_DOWNLOAD", "true")));
    Browser browser2 = playwright2.chromium().connectOverCDP(DEVURL);
    BrowserContext context2 = browser2.newContext(new NewContextOptions());
    Page page2 = context2.newPage();
    page2.navigate(HTML);

    page2.emulateMedia(new Page.EmulateMediaOptions().setMedia(Media.PRINT));
    // True
    System.out.println("page2 has print media type after setting media type to PRINT: " + isPrintMediaType(page2));
    // Should be True
    System.out.println("page1 has print media type after setting media type to PRINT: " + isPrintMediaType(page1));

    playwright1.close();
    playwright2.close();
  }

  private static String isPrintMediaType(Page page) {
    return page.evaluate("() => matchMedia('print').matches").toString();
  }
}


COVID KMI Communication #8 - 4 Versions

Value Statement
As a Veteran and non Veteran apart of expanded eligibility
I want to receive an informational email after I expressed interest in receiving information about the COVID vaccination process at the VA
that is customized to my current vaccination status
as well as my survey response
So I can be informed about the VA's plans for COVID
Acceptance Criteria
GIVEN a Veteran and non Veteran apart of expanded eligibility has expressed interest via the COVID Vaccination Interest Form(s) on VA.gov and the VA has information to communicate
WHEN VA.gov sends a request to VEText API
THEN VANotify will send an informational email to the Veteran and non Veteran apart of expanded eligibility
THAT is customized based on their response and current vaccination status
Checklist

[x] Receive draft content
[x] Create 4 template ID(s) in staging for COVID Information Update
[x] Create 4 template ID(s) in production for COVID Information Update
[X] Give the template ID(s) to Rob
[X] Receive approved content

[X] Update templates w/ approved content

[ ] HOLD FOR NOW - Remove the unsubscribe language/link from Segments 1 and 2.
[ ] Give Granicus a heads up
[x] Update Govdelivery weight to 75 and SES to 25 in prod

Assumptions

This is apart of VA.gov and VetsAPI
We need to segment the email per the below:

S1: "The Gap": Folks who signed up between when SLA first started circulating & passed until we had new KMI in place. We want to message them - hey! we need a bit more info from you. This would also include anyone who signed up early on who wasn't a registered veteran.
S2: Welcome! Folks who have signed up since the law passed (MSG: we have your info - your facility will be in touch. nothing you need to do yet!) -- Fairfax county style.
 (We're thinking this is a "last message for now" kidn of thing, so we don't continue to spam them, but if we need to reach out to them in the fall for a booster, we have them.)
Additional Info/Resources
https://app.zenhub.com/files/257953872/3a5a6a3f-ffea-48ca-abe3-f38cb8d18d0f/download
Out of Scope

Two-way communication with Veteran

Open Questions

Staging:

segment 1 - a7c0906d-55e3-4012-ba86-f225df049335
segment 2 - b6d2a96c-b29c-49a9-bfaf-c048ba4f0107
segment 3 - 05044bf6-8982-47bb-92e9-cb90c3168043
segment 4 - 027c1614-ae46-478a-b0fa-cf02150ec8bb

Production:

segment 1 - dd42aa9f-1309-4091-896c-2d1c5a468f59
segment 2 - 8f14e772-c5e2-4ae4-b40a-4ac737c06ec4
segment 3 - 501e0761-dea0-4eb3-84fe-9153ecf61359
segment 4 - 1402bb46-7b30-42e9-b710-a5e4a289a859

updated all templates

Updated COVID KMI content. https://app.zenhub.com/files/257953872/b9fa805f-5465-4c1e-9651-526fca4b6938/download

updated in staging and production

Tips for Veterans: Can we double check if there is an extra space between 2021 and the comma in the first sentence?

Need to fix the subject line here

updated subject line for s2

@emilywu-tw updated content for Segments 1 and 3 including info on the J&J vaccine. Let me know if you want to up on to discuss. https://app.zenhub.com/files/257953872/1b1cf6c6-ceeb-43ef-b275-682b211fc1d6/download

templates updated

We sent out segments 1 and 3 on 4/14. But still need to send out 2 and 4. Maria pinged Danielle for information on plans.

Feedback from client on Segment 4 "The bullets on this one look small to me too, so maybe it's just a forwarding thing. The header for the J&J alert needs to be bolded though." Attached screenshots.

After we fix the header, let's send the emails to them directly.
Because I think the bullet size is indeed a forwarding issue.

Here are the emails to send direct too:
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>

jnj header has been updated and emails sent!


Open and Export buttons nonresponsive

I am unable to produce any output from the Export or Open buttons in the query results pane. The buttons don't do anything at all. This seems to be specific to the SAP HANA driver for SQLTools as I have no problem exporting with other drivers like MySQL.

@farmer-in-a-tweed-cap I'm having the same issue. It seems like this driver is not maintained anymore :-(.


ghc: enable parallel building of GHC on aarch64.

Note: this only affects GHC, not haskellPackages, which still suffers
from https://ghc.haskell.org/trac/ghc/ticket/15449.
Motivation for this change
@ElvishJerricco pointed out on #47901 that GHC's own build system (mostly) doesn't use -j. (This is not quite true as it does seem to use it for building some phases, but my test build made it through successfully, anyway, so perhaps these phases don't trigger the GHC bug mentioned above in the Trac issue.)
Therefore, we can re-enable parallel builds of GHC on aarch64 and produce a working ghc843 in just a few hours on the Packet.net hardware. This will allow Hydra to start compiling haskellPackages and we can deal with aarch64 porting issues as they arise. In attempting to build my own Haskell packages with this compiler, many important packages (e.g., lens) build just fine, but tls's tests are currently broken on this arch.
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[x] Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[x] Fits CONTRIBUTING.md.

@dhess I wonder if there's a way to control the number passed to -j in the build process, without affecting the number passed to make. It'd be great if we could make all the ghc --make calls avoid -j categorically.

But overall I'm happy about this change :)

The GHC build system uses -j with --make all over the place. Reverting this changes breaks the GHC build on my aarch64 boards. As discussed on the ticket, this doesn't afflict all microarchitectures, but if you are affected, whether or not the build succeeds is a dice roll.
I would much rather have a build that works every time, rather than a build that fails on my machines but works on Hydra. Maybe I'm the only one unlucky enough to be using the afflicted hardware?

@TravisWhitaker I thought -j and --make are only used in building ghc-cabal and running the test suite. Where else is it used? We should probably disable the test quite though, unless we can figure out how to limit the -j for just that.

@TravisWhitaker The entire community benefits from having successful Hydra builds of GHC on aarch64, so I think that's the right tradeoff, even if it causes an inconvenience for you, personally. Though it's a pain, you can override ghc843 for your own builds with an overlay.
However, the point is moot (for now) as it appears that even Hydra is unable to build ghc843 reliably on aarch64 since the initial successful build:
https://hydra.nixos.org/build/83144215
It's timing out after 10 hours, which is interesting, because before this change, when ghc843 was being built with enableParallelBuilding disabled, it always took > 10 hours and therefore Hydra killed it mid-build. (Last I checked, the NixOS Hydra is configured to kill builds after 10 hours.)
The first successful build (https://hydra.nixos.org/build/82810083), just after this commit was made, took 8h23m, so it made the cut. But it might be the case that we just got lucky with that build, and that subsequent builds with enableParallelBuilding enabled have triggered the bug, causing the build to hang and Hydra to terminate it after 10 hours.
It's also possible that the builds were going fine but just took longer than the first successful build and didn't make the 10 hour cutoff.
So, either way, it appears that this change was not successful in its goal of producing consistent GHC Hydra builds on aarch64.
 These daily builds are working fine, so I'm not sure why they're succeeding when the official NixOS ones are not. I'll look into it.

The entire community benefits from having successful Hydra builds of GHC on aarch64, so I think that's the right tradeoff, even if it causes an inconvenience for you, personally.

With enableParallelBuilding = true, this derivation is nondeterministic in a critically bad way. If a derivation needed to be made nondeterministic to enable Hydra builds in a more benign way (i.e. something less than memory corruption), I'd certainly agree. However, unless these GHC binaries are passing validation (IIRC full validation isn't run, but perhaps it is) or we know for sure that the microarchitecture Hydra is using to build these is unaffected, I wouldn't trust the binaries in these packages.
I should've qualified "my" more carefully; my organization has its own Nix infrastructure and GHC tweaks, so we aren't using these binaries anyway. In general, I think a person with afflicted hardware would rather wait for a working binary to build, rather than download one from Hydra that might be arbitrarily broken due to this GHC bug.

It's timing out after 10 hours, which is interesting, because before this change, when ghc843 was being built with enableParallelBuilding disabled, it always took > 10 hours and therefore Hydra killed it mid-build.

As I wrote in the GHC bug report, a common failure mode is a threaded Haskell program hanging forever, waiting on a futex. Perhaps this is the culprit?

You might be running on unafflicted hardware (which seems to be any microarch without dynamic execution) or you've flipped three heads in a row.

I thought -j and --make are only used in building ghc-cabal and running the test suite. Where else is it used? We should probably disable the test quite though, unless we can figure out how to limit the -j for just that.

IIRC it's also used when building the boot packages, but I could be wrong.

@dhess I wonder if we could get Hydra to build it with https://github.com/NixOS/nixpkgs/pull/45744 so we can see how long each phase is actually taking.

@ElvishJerricco Does it always panic? I was under the impression that it sometimes just hangs indefinitely. tmobile on the GHC Trac, who originally filed the issue (https://ghc.haskell.org/trac/ghc/ticket/15449), lists one of the manifestations as, "Compiler process sleeps indefinitely."

@dhess Ah right. But it still wouldn't get to the installPhase or fixupPhase in that case. It'd hang in buildPhase or checkPhase or something.

It can't be because then we wouldn't be getting to the installPhase, let alone the fixupPhase.

Ah, agreed. It seems something else is going on...

@dhess Do the builders have auto-optimise-store turned on? I figure for an output as large as GHC, the post-build optimization of the path could potentially take a while on a slow core.

Does 8.4.4 support cross compiling from aarch64 to another architecture? I am getting some weird errors from cross-trunk:
https://hydra.nixos.org/build/83688270/nixlog/1/tail
ghc-stage1: panic! (the 'impossible' happened)
  (GHC version 8.4.4 for aarch64-unknown-linux):
        padLiveArgs -- i > regNum ??
CallStack (from HasCallStack):
  error, called at compiler/llvmGen/LlvmCodeGen/Base.hs:194:27 in ghc:LlvmCodeGen.Base

Actually this doesn't seem to be specific to aarch64, so probably not having to do with this PR.

Indeed, GHC 8.4.4 is totally broken on ARM.
https://ghc.haskell.org/trac/ghc/ticket/15780

I should have more time to look into this in a couple of weeks, but I'm completely slammed at the moment. Meanwhile, as recently as master from 4 days ago, ghc843 continues to work great on the NixOS aarch64 community builder, somehow.


Unhandled Promise Rejection: TypeError: Cannot read property 'pid' of undefined

Describe the bug
Getting this error every time I exit the app
To Reproduce
Steps to reproduce the behavior:

Open app
It says "Waiting for smesher response..." for a very long time (hours), despite the node being fully synced (#501)
Click the "X" to close the app - no prompt (#502), it immediately begins to shut down, closes the underlying node, but then hangs

Expected behavior
Expected the app to shutdown correctly
Screenshots

Desktop (please complete the following information):

OS: Linux

Full error text:
TypeError: Cannot read property 'pid' of undefined
    at n (/opt/Spacemesh/resources/app.asar/desktop/main.prod.js:2:17894)```

Possibly related to #501 and #502, since all three issues seem to be occurring together.

fixed in #495
bug was caused by trying to kill node process before it actually started


Disables owner checks for clustered ETCD

How to categorize this PR?

/area high-availability
/area control-plane-migration
/kind enhancement
What this PR does / why we need it:
This PR disables owner checks (initially added by https://github.com/gardener/etcd-backup-restore/pull/383) for the etcd-main Etcd resource when the desired replicas are greater than 1. Replicas greater than 1 basically means that the HAControlPlanes feature gate is enabled and the shoot is annotated with alpha.control-plane.shoot.gardener.cloud/high-availability, and therefore the etcds for the shoot will be started as a cluster.
Which issue(s) this PR fixes:
Part of #6302
Special notes for your reviewer:
/cc @ishan16696 @timuthy @unmarshall
When the owner checks are not specified in the Etcd resource, etcd-druid additionally takes care of deploying the etcd StatefulSets with the following readinessProbe:
readinessProbe:
  exec:
    command:
    - /usr/bin/curl
    - --cert
    - /var/etcd/ssl/client/client/tls.crt
    - --key
    - /var/etcd/ssl/client/client/tls.key
    - --cacert
    - /var/etcd/ssl/client/ca/bundle.crt
    - https://etcd-main-local:2379/health

When testing this PR locally I noticed that the etcd-main and etcd-events pods never get ready with the following error:
  Warning  Unhealthy  61s   kubelet            Readiness probe failed:   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
curl: (7) Failed to connect to etcd-main-local port 2379 after 0 ms: Connection refused
  Warning  Unhealthy  0s (x12 over 55s)  kubelet  Readiness probe failed:

Release note:

Owner checks (which are used by the `backup-restore` sidecar to determine whether the owner domain name resolves to the specified owner ID and if not, take a final full snapshot and disable the cluster), will no longer be enabled by `gardenlet`, if the `HAControlPlanes` feature gate is enabled, the `Shoot` is annotated with `alpha.control-plane.shoot.gardener.cloud/high-availability` and the `Shoot`'s ETCDs are started as a cluster (with more than 1 replica).

@plkokanov is it related to test failures of #6411 and needs to go in v1.52?

@timuthy Sorry, I wasn't clear enough - I know that the change to /health is expected behaviour and intended.
I merely added the error as an observation and am currently collecting a few more logs.
@acumino no this PR should not be related to the PR you mentioned and can does not have to be in 1.52

I had an offline discussion with @plkokanov first to clarify the responsibilities about owner checks. We came to the conclusion that it's Gardener's domain knowledge if the owner check is supposed to be set or not.
Today, also Etcd-Druid has this knowledge to skip owner checks for multi-node etcds which is an undesired duplication.
Even if this PR does not fix an issue per se, we can continue with the explained approach. I'll have a look at the changes beginning of next week.

/retest


glibcLocales: Fix build for duplicates in locales list

Motivation for this change
Without this patch, setting the same locale twice, e.g. like this in
NixOS:
{
  i18n.supportedLocales = [
    (config.i18n.defaultLocale + "/UTF-8")
    (config.i18n.defaultLocale + "/UTF-8")
  ];
}

Would make the glibcLocales build fail with
Error: unsupported locales detected:
en_US.UTF-8/UTF-8 \
You should choose from the list above the error.

Ping @Mic92 @Ericson2314 @vcunat
Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
21.11 Release Notes (or backporting 21.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[ ] Fits CONTRIBUTING.md.

Would it be a better idea to do that in the nix code that validates the option entries?

@SuperSandro2000 I think this solution is a bit better since it fixes the problem directly where the uniqueness of the list is required (for the comm command). And if we only fixed it on the NixOS module level, you could still get the error if you are using pkgs.glibcLocales directly.
For the future I think we should definitely have a types.set or similar for NixOS modules, because there's many such situations where it's really a set that we want, not a list.

I think I need a set type for properly merging pipewire modules. Right now they are also duped.


Polyfill locales not getting copied to output folder

Hey,
we're using ember-intl 2.6.1. When we started using it all locales where copied to assets/intl/locales/. We changed locales to only include en-us and de-de. Back then this worked as well. But somewhere along the way something broke.
Now not even then ´locales´ folder is created during build. Deleting my output and my tmp folder does not help. The intl folder with its libs is there as usual. When I manually copy the locale files from node_modules/intl/locale-data/jsonp everything works as expected.
Here is the content of config/ember-intl.js:
/*jshint node:true*/

module.exports = function(environment) {
  return {
    locales: ['en-us, de-de'],
    baseLocale: 'en-us',
    disablePolyfill: false,
    publicOnly: false,
    inputPath: 'translations',
    outputPath: 'translations'
  };
};

The build process doesn't give me any errors.

Can you verify this goes away with 2.10.x?

Hey, sorry for the late feedback. 2.10.1 fixed this! Curiously it also logged the following warning during build:
[ember-intl] 'en-us, de-de' is not a valid locale name

Turns out I forgot the string delimiters in my ember-intl.js from above. Totally missed that. After correcting it it also worked again in 2.6.1.
Sorry for the inconvenience!

Thanks for reporting back and no issue, glad I threw those warnings in :)


Implemented strict DI & separating controller logic

The original directive would have failed in cases of strict DI (dependency injection) or minification. It failed for me while using it on a Firefox OS Flame 2.0 device, while it  worked on simulator. Even with minification, (with implicit DI) it would replace the "$scope" function argument with some variable like "b". This would fail inside the controller logic since it would expect $scope, but gets "b".
Refactored the main script file to do two things:

Implemented changes to have explicit annotation for supporting strict DI (minification would work).
Separate out controller logic from the directive. This keeps the directive free of any business logic.

With this, now the directive can be used under strict DI scenarios as well as minification. Please check.
Awesome widget btw! Thanx!

Thank you.
Your PR is out-dated. The minification issue was recently fixed. Can you please try with the latest repo.

That's true, my teammate was using an older version of your script. I checked the latest one, its good. However, it still has a scope of improvement:

Separate out controller logic from the directive. This keeps the directive free of any business logic.

That would make your code modular as well, since its best to keep the directive content to a minimal, with just the restrict & template params (with other misc), but not controller logic. See if you find that useful and convincing enough for merge. Just wanted to help you get your stuff better.

You are absolutely correct. I will merge it, but, your PR still requires a reverse merge in order to be merged into master. Thanks.

Sorry, I am relatively new to Git. Don't know what reverse merge means. Is there anything specifically I need to do?

It means you need to update your PR with the latest master branch. The master branch was in StateC (for example) when you pulled it and started working on it. Meanwhile other changes were pushed into the master branch and now lets say it is in StateH. Now your PR is missing the changes pushed into master while it transitioned from StateC to StateH. You need to pull the master repo into your working repo. Resolve the conflicts and then push again, so, it can be merged directly into master branch.

Ok, I will close this one and create a new pull request using the latest version of the code.


Order talks consistently by track per time slot

This is probably easier to do as an API change (might move the issue over there if I start working on it and find that this is the case) but when viewing talks in the list view on e.g. a four-track conference, for two slots where all four tracks are occupied, the third talk in the list for each slot should correspond to the same track, vs. relying on talk insertion order (which gets extra messy for subbing in talks later).
Eventually we'll want to allow specified order for tracks. But relying on track insertion order (AKA PK) should be sufficient for now, as tracks take less elbow grease to create than talks.

Looks like this will be slightly easier to perform in this repo; there's an N+1 going on in the talk list for grabbing tracks so it's not really any more efficient to sort talks by track (which is a 1:M somehow) API-side than client-side.
Just checked the database and we actually only ever have one track associated per talk, though some talks have a track ID of 0 multiple times. So we could actually pull the track ID into the talk model as an ?int, cleaning up both existing data and querying in the process.