Add CancelGracePeriod param to aws template

It seems this value isn't modifiable via environment variables, so it would be nice to be able to modify this value in the cloudformation config

Thanks for the PR @chrisdothtml (and I'm also looking into the broader issues you're having with cancel-grace-period).
I'm a bit nervous about adding a CloudFormation parameter for this (and potentially every other) agent configuration value. I'm wondering if we should bring in some other way to override agent configuration. For example an SSM Parameter Store namespace could be specified, and the values therein could be written to the agent configuration during boot. Do you think that would work for your use-case? (any thoughts @yob @chloeruka @lox?)
If we do stick with this CloudFormation parameter, I believe you'll need to update packer/linux/conf/bin/bk-install-elastic-stack.sh and packer/windows/conf/bin/bk-install-elastic-stack.ps1 to explicitly read it from env and write it to the config.

I share the concern about a parameter count explosion if we expose all the agent flags, but it's also not a great situation at the moment. Changing flags with a bootstrap script is possible, but super awkward.

For example an SSM Parameter Store namespace could be specified, and the values therein could be written to the agent configuration during boot. Do

I'm not familiar enough with SSM Parameter Store to know the tradeoffs involved here, but at face value it sounds like a neat solution with lots of flexibility for the future.

The main trade-off I can think of, and not specific to Parameter Store, is that the parameters/config would be outside the CloudFormation model. Machines would pick up whatever config was set at their individual time of boot, rather than what is defined in the stack itself. So if the parameters are changed over time, a stack could end up with different config on different machines. I don't really see this as a big problem, though, and sometimes it might be advantageous.

Agreed that doing agent configuration in the bootstrap is super awkward and that adding CloudFormation params for every agent config key would be really noisy. How about instead of an SSM param store you could just specify an S3 url for the agent config file, similar to the existing mechanism in BootstrapScriptUrl? Something like AgentConfigUrl

One complication with both the SSM and S3 approach is that there's a handful of config options that maybe do need their own stack parameter (like agent token and disconnect-after-idle-timeout) or arguably shouldn't be configurable at all (like *-path). I guess we could explicitly exclude/override those values with stack parameters?

The SSM/S3/whatever config (let's call it “user config” for now) would certainly need to be merged somehow with the base config file and the stack-driven config keys. But maybe the user config can just take highest precedence, and it's up to the user to not set values that are meant to be controlled elsewhere?
One benefit of SSM Parameter Store over S3 is that people could more easily use an outer CloudFormation / Terraform stack to manage the user config key/values, if they wanted. But it's true that a file in S3 would be more fitting with the existing elastic-stack tech, and perhaps more approachable to people unfamiliar with SSM.

@pda Would you be amenable, if I or @yob continue this PR, to overlooking the possible slippery slope toward exposing all the configuration values (which I think we all agree is suboptimal)? Might we be able to add support for adjusting (only) this agent configuration value (using the existing system), first, before we look at different ways of configuring all the other useful agent configuration values (which are...?)
One reason, in particular, that cancel-grace-period should be tunable (before/as opposed to making everything tunable) is that the default is deficient for a number of scenarios (e.g. in mine, it cancels Terraform runs mid-flight and leaves a lock file behind). You can see this in how the original PRs/issues talk a lot about 30 second or two minutes being a better default than 10 seconds, and how lox mentions it should probably be configurable per-step
I just think worrying about needing to expose all the configuration values is probably premature. YAGNI: I've not needed anything except cancel-grace-period (or things already exposed via CloudFormation parameters) in all the time we've been using the AWS stack (years).

For ease of testing: #881 is this PR rebased, with the fixes that were asked for to the bk-install-elastic-stack scripts (to write out the config value). Did this mainly so I can use it myself. Although, I'm curious, @yob said:

Changing flags with a bootstrap script is possible, but super awkward.

Could you describe how? Maybe I can use that? Do you have to kill the agent process that's running the bootstrap script and relaunch it somehow? Is there a signal to get it to re-read its configuration?
AFAICT by testing, by the time the bootstrap script is running, it's too late for most sensible methods to set this flag. So all that's left is custom cloud-init code, which is another level of difficulty? Is there an easier way?

Could anyone describe how? Maybe I can use that? Do you have to relaunch the agent process somehow? Is there a signal to get it to re-read its configuration?

The BootstrapScriptUrl parameter is executed on every instance boot after the agent config is written but before the agent is started. So you should be able to append settings to the config file using the bootstrap script before the agent process ever starts up.

append settings to the config file using the bootstrap script

Brilliant, the missing link was that we always tried to set environment variables using the bootstrap before. A pretty easy workaround after all!
echo "cancel-grace-period=30" >> /etc/buildkite-agent/buildkite-agent.cfg

https://github.com/buildkite/elastic-ci-stack-for-aws/pull/1258


Kleiner Darstellungsfehler im Papierkorb

Seems like quotation marks are not encoded

yup :+1:

also translation doesn't work


[ListItem] Clear hover state if componet get's disabled

Issue is anlog to https://github.com/callemall/material-ui/pull/4531
I had the same problem with ListItems instead of FlatButtons
https://github.com/callemall/material-ui/pull/4531#issuecomment-259443648

[x] PR has tests / docs demo, and is linted.
[x] Commit and PR titles begin with [ComponentName], and are in imperative form: "[Component] Fix leaky abstraction".
[x] Description explains the issue / use-case resolved, and auto-closes the related issue(s) (http://tr.im/vFqem).

Thanks for working on this issue. That looks good. However, could you add a unit test for this new piece of logic?

@Pczek Thanks!


Schedule the merge conflict labelling to run

Runs at 4:20 UTC every day
It was taking a while for labels to get removed, sometimes

If you want to test this, dont commit anything to main until 4:20 tomorrow, when https://github.com/R2Northstar/NorthstarLauncher/pull/617 should lose the label

Merged and should run within the next minute ^^


[SuperReader] setState() called after dispose

I use SuperReader widgets in order to display a document, but the document itself is tappable and pushes a new route. The following issue has been happening quite often recently:
════════ Exception caught by scheduler library ═════════════════════════════════
The following assertion was thrown during a scheduler callback:
setState() called after dispose(): _ReadOnlyIOSDocumentTouchInteractorState#5e160(lifecycle state: defunct, not mounted, ticker inactive)

This error happens if you call setState() on a State object for a widget that no longer appears in the widget tree (e.g., whose parent widget no longer includes the widget in its build). This error can occur when code calls setState() from a timer or an animation callback.

The preferred solution is to cancel the timer or stop listening to the animation in the dispose() callback. Another solution is to check the "mounted" property of this object before calling setState() to ensure the object is still in the tree.
This error might indicate a memory leak if setState() is being called because another object is retaining a reference to this State object after it has been removed from the tree. To avoid memory leaks, consider breaking the reference to this object during dispose().

When the exception was thrown, this was the stack
#0      State.setState.<anonymous closure>
framework.dart:1097
#1      State.setState
framework.dart:1132
#2      _ReadOnlyIOSDocumentTouchInteractorState.didChangeDependencies.<anonymous closure>
read_only_document_ios_touch_interactor.dart:194

I think there should be a mounted check before setState is called. This is in didChangeDependencies in ReadOnlyIOSDocumentTouchInteractor.

It would be easier if you can provide more detail about the code that shows this error?

@matthew-carroll It seems this is already solved, because now we are using onNextFrame, which already checks if the widget is mounted.

@Jethro87 I'm going to close this based on Angelo's comment. Please re-open if the issue persists.


vkGetDeviceQueue is not strong enough

The vkGetDeviceQueue is not strong enough, when the queueFamilyIndex or queueIndex is out of range, there's only null accessing message generated but no more information:
0xC0000005: Access violation reading location 0x00000000 (call stack: _vkGetDeviceQueue@16())

I think the vkGetDeviceQueue should not modify the value of pQueue when the index is out of range, instead of null accessing.

Are you seeing this error while running the Validation layers?

I disagree. If you are running validation then the behavior should not change the behavior unless you tell it to do so. I would also have expected param checker to throw an error in this case so I'm guessing you are not in which case I also don't think the driver should be responsible for this as one of the major points of the layers is to catch app issues so the driver doesn't have to.

Enabling standard validation should produce the following errors when the queueIndex is out of range:

ERROR: [PARAMCHECK] Code 1 : VkGetDeviceQueue parameter, uint32_t queueIndex 99, must be less than the number of queues given when the device was created.
ERROR: [DL] Code 9 : Invalid queue request in vkGetDeviceQueue(). QueueFamilyIndex 0 only has 1 queues, but requested queueIndex is 99.

You can find instructions for enabling validation layers in the layers README file.

Closing due to inactivity


Add support for Google PaLM2 models

This will be added with this issue:
https://github.com/Helicone/helicone/issues/464


gpt4all not running when click open button

GPT4All will run if you type gpt4all at the Tea prompt, but does not startup when you hit the Open button in the Tea Gui. It just opens to the Tea prompt.
This happened with auto-gpt as well and just wanted you to be aware that the command isn't being run after clicking Open in the GUI.

On another note, when I run gpt4all at the prompt, it only runs the CLI version of the application, but the screens and descriptions are for the web-based version.

@clickbrain thanks for reporting this we will take a look

gpt4all has no entrypoint as yet. Easy enough to add. pkgs without entrypoints only open a terminal with the pkg env loaded


What happened with this?

VSCode Version: 1.40.1
OS Version: Win10 PRO 64x - Build 1903

Steps to Reproduce:

When I started giving this, I formatted the computer, stopped and then started again.

Does this issue occur when all extensions are disabled?: Yes/No

I don't know.

That looks like corrupted video drivers or a bad card to me.

That looks like corrupted video drivers or a bad card to me.

Thx, i'll check if is card video.

You can also try to run VS Code with --disable-gpu flag from console or other ways. See https://code.visualstudio.com/updates/v1_40#_disable-gpu-acceleration

Already fixed with "Driver Card Update"
I have unistall and reinstall all again and all is fixed! Thanks for all here. :}


[Event Request] Codeunit 80 "Sales-Post" - OnBeforeUpdateInvoicedQtyOnShipmentLine

Hi, would it be possible to get an OnBeforeUpdateInvoicedQtyOnShipmentLine event in codeunit 80 (Sales-Post), like this:
    local procedure UpdateInvoicedQtyOnShipmentLine(var SalesShptLine: Record "Sales Shipment Line"; QtyToBeInvoiced: Decimal; QtyToBeInvoicedBase: Decimal)
    begin
        OnBeforeUpdateInvoicedQtyOnShipmentLine(SalesShptLine, QtyToBeInvoiced, QtyToBeInvoicedBase);  // New Event
        with SalesShptLine do begin
            "Quantity Invoiced" := "Quantity Invoiced" - QtyToBeInvoiced;
            "Qty. Invoiced (Base)" := "Qty. Invoiced (Base)" - QtyToBeInvoicedBase;
            "Qty. Shipped Not Invoiced" := Quantity - "Quantity Invoiced";
            OnUpdateInvoicedQtyOnShipmentLineOnBeforeModifySalesShptLine(SalesShptLine, QtyToBeInvoiced, QtyToBeInvoicedBase);
            Modify;
        end;
    end;

 [IntegrationEvent(false, false)]
    local procedure OnBeforeUpdateInvoicedQtyOnShipmentLine(var SalesShptLine: Record "Sales Shipment Line"; QtyToBeInvoiced: Decimal; QtyToBeInvoicedBase: Decimal)
    begin
    end;

Thanks!

Thanks for reporting this. We agree, and we’ll publish a fix asap, either in an update for the current version or in the next major release. Please do not reply to this, as we do not monitor closed issues. If you have follow-up questions or requests, please create a new issue where you reference this one.


SwapVecDeque

When working with FIFO/LIFO access patterns one might only want to keep the last x front/back elements in memory. This may have great performance while still keeping memory low.
The concrete issue I am trying to solve is that single postgres queries for such a access pattern are still too slow in my case.

I found that MacOS automatically switches to swap memory in case my rust app consumes ton of memory.


feat: allow relabel config for servicemonitors

Allows specifying relabel configs for the service monitors. Might be helpful, for some?

i am gonna redo this


DRILL-5660: Drill 1.10 queries fail due to Parquet Metadata "corrupti…

…on" from DRILL-3867

bump up metadata file version to v3_1;
ignoring of unknown metadata version (for example metadata generated from future versions of drill);
adding removing "%20" symbols from path.

I wonder if you are very close to solving another long-running problem in Drill: Parquet metadata file corruption (or, at least, the corruption causing queries to fail.)
We have an issue that two Drillbits can write to metadata file concurrently and corrupt it. We also have a case where Drillbit A writes the file while Drillbit B reads it. In both cases, the reading Drillbit can't deserialize the file and "bad things happen."
We need code that can try to deserialize a file, notice that the JSON in the file does not fit our schema, and handle the deserialization exception. If we do that, it is a small step to ignore all forms of read errors: file-not-found, file truncation, etc.
In this case, we give an explicit version error if we can read the file and it has the wrong version. If we can't deserialize the file, we log an error about "file is of a newer version or corrupt."
After this, the corruption can still occur, but the user view is that queries run slow instead of just failing.

@paul-rogers could you please review additional commit. The main changes:

All IOExceptions are ignored while deserializing metadata cache files (JsonMappingException, JsonParseException...) with appropriate logging. To avoid of reading such corrupted or unsupported file again that status is stored in metadata context.

Creating the metadata context while expanding Selection from Metadata Cache (if it is null) allows to always detect the status of partition punning.

Two new test cases are added: testCorruptedMetadataFile() and testFutureUnsupportedMetadataVersion().

Test data created in @BeforeClass in TestParquetMetadataCache will be removed after executing all tests.

For  testMetadataCacheAbsolutePaths() and testSpacesInMetadataCachePath() tests  testPlanMatchingPatterns() is added.

@vdiravka
limiting minor version to 9 is breaking the whole concept of major and minor versions usage. Since when we reach minor max we'll need to bump up the major though structure changes did not occur.
If using float is not working for major and minor version (I believe you having trouble comparing 1.1 and 1.10) then let's leave our MetadataVersion class, just get rid of v in version name.

@arina-ielchiieva Small fix was made to resolve some regression tests failings. The branch is rebased to the master version.

+1, LGTM.


JSON-RPC: Improve basic and advanced search

This is a tweaked and more minimal version of #2723

I've tested it thoroughly to ensure "quick" searching is done using an OR condition.
I've made the libraryID and itemType conditions required so the don't get ignored when using an OR condition.
I've removed the citationKey condition from the advanced search as it can result in unexpected / undesired results. My logic here is that when using advanced search, the API user likely knows what their doing and could manually add ['citationKey', 'contains', 'whatever'] if so desired.

Let me know what you think of this. @f3fora as well!

lgtm! @f3fora?

For me, this actually do not solve what my PR wanted to solve. There are some constrains in advance search(ignore feeds and ignore attachments) which cannot be removed.
Ofc, adapting the quickSearch code is a good way approach to search into title creator year OR citationkey.
I applied this approach to a new commit of my PR.

For me, this actually do not solve what my PR wanted to solve. There are some constrains in advance search(ignore feeds and ignore attachments) which cannot be removed.
Ofc, adapting the quickSearch code is a good way approach to search into title creator year OR citationkey.
I applied this approach to a new commit of my PR.

Super -- merged that, thanks! @mgmeyers, if yours still does something else, can you rebase so we can discuss those changes?


Update balance on senderId change

closes #120

Added the NFT types for better component rendering

superseded: https://github.com/cennznet/ui/pull/127


Error [ERR_WORKER_OUT_OF_MEMORY]: Worker terminated due to reaching memory limit: JS heap out of memory

Describe the bug
When I use the "npx @wakaru/cli unminify" command in my js file, it gives the following error. The computer I use is extremely powerful. What should I do? I've added the relevant file below.
"Since the javascript file was not loaded, I set the extension to txt"

main.txt
Input code
No response
Reproduction
No response
Steps to reproduce
No response
Expected behavior
no memory error
Actual behavior
No response

@Yusufkulcu Dumb question.. but did you try it again following the instructions in the warning message shown in your screenshot about how to potentially resolve out of memory issues by using the --max-old-space-size?
If you googled "node js heap out of memory" or similar you would also turn up results like this, which also talk about using --max-old-space-size:

https://stackoverflow.com/questions/38558989/node-js-heap-out-of-memory
https://medium.com/@fatma_2377/understanding-and-resolving-the-fatal-error-javascript-heap-out-of-memory-issue-aac8da6b73fe
etc

The solution is to follow the instructions in the message. I think the instructions are not step-by-step, so let me give one here.
MacOS / Linux
NODE_OPTIONS="--max-old-space-size=8192" npx @wakaru/cli unminify

Windows (CMD)
set NODE_OPTIONS=--max-old-space-size=8192 && npx @wakaru/cli unminify

8192 in the command is 8GB. You can adjust the number based on your PC's specs and the size of the file.
BTW, what is the size of your input file?

The solution is to follow the instructions in the message. I think the instructions are not step-by-step, so let me give one here.
MacOS / Linux
NODE_OPTIONS="--max-old-space-size=8192" npx @wakaru/cli unminify

Windows (CMD)
set NODE_OPTIONS=--max-old-space-size=8192 && npx @wakaru/cli unminify

8192 in the command is 8GB. You can adjust the number based on your PC's specs and the size of the file.
BTW, what is the size of your input file?

The file size is around 6-7mb. I'm trying the suggestion you said. How do we need to give the 8gb value you specified?

How do we need to give the 8gb value you specified?

@Yusufkulcu I'm assuming you mean 'why do we need to specify 8gb if the file is only 6-7mb'? If so, it's because the process of unminimising the code needs to parse the plain text of the original file into a number of objects and data structures that are held and manipulated in memory, which causes it to expand to a much larger size during the process; and as the StackOverflow link I sent suggested, node has a default limit of only 1.7GB or similar.

I will try to expand the instructions or link them to README to check the detailed guide.

The file size is around 6-7mb. I'm trying the suggestion you said. How do we need to give the 8gb value you specified?

@Yusufkulcu I give the 8GB based on your mentioned "powerful PC." Based on my experience, 6-7 MB should only need ~4GB.


Events fix.

This should fix this error. https://github.com/illuspas/Node-Media-Server/issues/32
Basically, if you're going to deference the session and it's socket, you don't need to manually remove events, garbage collector will do it for you, as far as I know.
But anyway, after some tests I found out the true reason of the crash. As stated in the issue, crash only happens if the client can't keep up with the stream. So I put some console logs before every socket write and found out both places where it gets called right before the crash. And at this point in time of the code lifecycle, session is already closed, the socket is closed and all event listeners are removed. So this is not really a fix, just a workaround. Real fix would entail finding out the real reason why these sections of the code even get called.
Here are these places.
https://github.com/illuspas/Node-Media-Server/blob/0d5711334776763c6416e8a55569835e6162843e/node_rtmp_session.js#L603
https://github.com/illuspas/Node-Media-Server/blob/0d5711334776763c6416e8a55569835e6162843e/node_rtmp_session.js#L660

Made a small change to remove only data and close event listeners, but leave the error event listener in place to catch any errors.

This may be a compromise solution , if there is no better way I will merge pr.
I've been busy recently, so the reply will not be timely.

Yeah, it's more of a band-aid than fix. In the meantime I'll try to find a real fix for this issue.


Enabling pre-load of the Registry

Object uses an after BUILD modifier to trigger the population of the
registry and associated lookups. Use with a pre-fork server to
avoid nasty running server process startup costs caused by reaping
processes.

Please check this

Interesting. I may add a patch to load the Compara basic objects too

Ok sounds good. I assume you're talking about GenomeDBAdaptor & MLSS?

On 13 Nov 2015, at 17:20, Matthieu Muffato<EMAIL_ADDRESS>wrote:
Interesting. I may add a patch to load the Compara basic objects too
—
Reply to this email directly or view it on GitHub https://github.com/Ensembl/ensembl-rest/pull/93#issuecomment-156492830.

I am

should those changes also go into the production ensembl_rest.conf.default?


e2e: improve operator test

Please provide a description of this PR:

fail fast
reduce retryTimeOut to 5m, this will make test fail fast if there's a network issue.

LGTM and I didn't see any remarks in the files that the timeout was longer to fix a test flake for example. Will see if anyone else has any concerns.

@zirain: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

integ-pilot-istiodremote-mc_istio
2cebb3c614bc5d8de135bdcb426f33fa86a9307d
link
unknown
/test integ-pilot-istiodremote-mc

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@zirain: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

integ-pilot-istiodremote-mc_istio
2cebb3c614bc5d8de135bdcb426f33fa86a9307d
link
unknown
/test integ-pilot-istiodremote-mc

integ-pilot-multicluster_istio
2cebb3c614bc5d8de135bdcb426f33fa86a9307d
link
unknown
/test integ-pilot-multicluster

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

/retest

@zirain: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

integ-pilot-multicluster_istio
2cebb3c614bc5d8de135bdcb426f33fa86a9307d
link
true
/test integ-pilot-multicluster

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

/retest

@zirain: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

integ-pilot_istio
2cebb3c614bc5d8de135bdcb426f33fa86a9307d
link
unknown
/test integ-pilot

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@zirain: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

integ-pilot_istio
2cebb3c614bc5d8de135bdcb426f33fa86a9307d
link
unknown
/test integ-pilot

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.


[FEATURE]: Enhanced OracleDB Integration in Drizzle ORM

Describe what you want
As a Node.js developer working with Oracle databases, I frequently encounter challenges integrating Drizzle ORM with OracleDB due to limited documentation and compatibility issues. With enhanced OracleDB integration in Drizzle ORM, I envision a smoother development experience, better performance, and broader adoption of Drizzle ORM within the Oracle ecosystem.

Any news?

https://github.com/orgs/drizzle-team/projects/1/views/1?filterQuery=oracle

and also
https://github.com/drizzle-team/drizzle-orm/issues/289
has been moved to
https://github.com/drizzle-team/drizzle-orm/discussions/2442


Create a findings goal to reliably display issues found after a scan

The Metrics API is not something that should be called directly after the bom upload as a number of async tasks are executed to create the Metrics analysis result.
However the findings API is queryable after the bom upload and can provide details on the findings in the upload immediately.
Acceptance Criteria

Add a findings goal to reliably display vulnerable components after a bom upload
Document the new goal and provide recommended usage
Update metrics documentation to describe its recommended usage

Completed on v0.6.0


where clause on a nested document

Hi
I have a nested document like this :
{     "_id" : ObjectId("598577f3acb9ee0c3b3e7199"),     "shenase" : "12300",     "name" : "ff",     "vehicle" : "van",     "locations" : [          {             "time" : "12:16:59",             "date" : "12/05/1396",             "latiude" : 35.7380266088257,             "langtiude" : 51.5072907224076         },          {             "time" : "12:17:00",             "date" : "13/05/1396",             "latiude" : 35.7380266088257,             "langtiude" : 51.5072907224076         },          {             "time" : "12:17:02",             "date" : "14/05/1396",             "latiude" : 35,             "langtiude" : 51.5074229552436         }     ] }
Now I want to make  a where clause query like this but I dont know who should I do this .
here is the code that I tried but unsuccessful .
$locations = DB::collection('mycollection')->where("locations.latiude", "=", 35)->get();

Can't reproduce the problem:
php artisan tinker
>>> use Illuminate\Support\Facades\DB;
>>> DB::collection('mycollection')->where("locations.latiude",35)->get();
=> Illuminate\Support\Collection {#889
     all: [
       [
         "_id" => MongoDB\BSON\ObjectID {#888
           +"oid": "598577f3acb9ee0c3b3e7199",
         },
         "shenase" => "12300",
         "name" => "ff",
         "vehicle" => "van",
         "locations" => [
           [
             "time" => "12:16:59",
             "date" => "12/05/1396",
             "latiude" => 35.738026608826,
             "langtiude" => 51.507290722408,
           ],
           [
             "time" => "12:17:00",
             "date" => "13/05/1396",
             "latiude" => 35.738026608826,
             "langtiude" => 51.507290722408,
           ],


feat: 3712 - new mlkit and zxing scanners on flutter 3.7

New files:

smooth_barcode_scanner_mlkit.dart: Barcode scanner based on MLKit.
smooth_barcode_scanner_mockup.dart: Fake barcode scanner, for tests.
smooth_barcode_scanner_type.dart: Barcode scanner types.
smooth_barcode_scanner_zxing.dart: Barcode scanner based on ZXing.

Deleted files:

scanner: removed folder
camera_controller.dart
camera_image_cropper.dart
camera_full_getter.dart
camera_image_preview.dart
camera_modes.dart
lifecycle_aware_widget.dart
lifecycle_manager.dart
scan_flash_toggle.dart
scan_visor.dart
scanner_overlay.dart
user_preferences_dialog_editor.dart

Impacted files:

app_test.dart: now uses new enum SmoothBarcodeScannerType
background_task_badge.dart: now uses flutter 3.7 badge
basic_test.dart: now uses new enum SmoothBarcodeScannerType
build.gradle: upgraded kotlin to 1.8.0
camera_helper.dart: simplified
camera_scan_page.dart: simplified
constant_icons.dart: added an adaptive "flip camera" icon
file_cache_manager_impl.dart: minor 3.7 refactoring
goldens.dart: minor 3.7 refactoring
labeler.yml: removed references to deleted files
main_fdroid.dart: now uses new enum SmoothBarcodeScannerType
main_google_play.dart: now uses new enum SmoothBarcodeScannerType
main_ios.dart: now uses new enum SmoothBarcodeScannerType
network_config.dart: minor 3.7 refactoring
new_product_page.dart: minor 3.7 refactoring
onboarding_bottom_bar.dart: minor 3.7 refactoring
pubspec.lock: wtf
apple_app_store/pubspec.yaml: minor 3.7 refactoring
google_play/pubspec.yaml: minor 3.7 refactoring
shared/pubspec.yaml: minor 3.7 refactoring
uri_store/pubspec.yaml: minor 3.7 refactoring
data_importer/pubspec.yaml: minor 3.7 refactoring
data_importer_shared/pubspec.yaml: minor 3.7 refactoring
smooth_app/pubspec.yaml: now includes mlkit and zxing scanners; minor 3.7 refactoring; removed now redundant badge; minor upgrades
scan_page.dart: simplified
user_preferences.dart: removed now useless methods
user_preferences_dev_mode.dart: removed duration and scan parameter settings
user_preferences_settings.dart: removed now useless parameter; minor 3.7 refactoring

What

Heavy refactoring of the barcode scanners.
Basically, now we rely on existing barcode scanners: mobile_scanner for mlkit and qr_code_scanner for zxing.
En passant, upgraded to flutter 3.7.

Screenshot

mlkit
zxing

Fixes bug(s)

Fixes: #3712
Fixes: #3674
Fixes: #3670
Fixes: #3664
Fixes: #3521
Fixes: #3427
Fixes: #3349
Fixes: #3348
Fixes: #3347
Fixes: #2917
Fixes: #2849
Fixes: #2548

The code was written in flutter 3.7, as it was the main focus of the issue. The barcode scan refactoring was "only" a side-effect.
Therefore, the CI has to be upgraded to 3.7 / 2.19 for the pre-tests to work.
Meanwhile, we would be better off with a new official version of Smoothie - hopefully the last flutter 3.0.5 version.

Should I push the ci changes on your branch

Should I push the ci changes on your branch

Please do, thanks!

I'm making slow progress on pushing out the release to the Apple app store, we had an issue with the CI, which is not solved yet.

Ok, I've fixed the build, and the iOS build has been proposed to Apple.

First of all thanks for your work @monsieurtanuki

You're welcome!

We should implement some kind of on phone testing

I'm not sure how we could do that, if you mean automated tests.

Ci can be fixed to the new version with no problem, though cd is going to be harder, we likely can't build if we remove the ml or zxing dependencie, but let this be my worry in a follow-up pr.

Not sure what you mean, but as I said in a comment for this PR we may not have to select either zxing or mlkit, build wise: we just embed both and the only different is the entry point that selects which one.

We need extensive analytics

I'm not sure that's relevant anymore, as most of the work is done by the external packages.
Which analytics would you like to add?

When someone first starts the app (also after an update?) we hand the currently selected engine a mock image which a barcode, as close to how the scanner does it as possible. If it does not work, we report it to sentry/matomo? and switch the engine. I'd suggest to at first push both to play store to now break anything on edge devices.

Is that a fact? I believed that there was only one embedded scanner in the previous version.
In addition to that, that would make sense only for mlkit downgraded to zxing, as - for some reason - on FDroid mlkit is "forbidden".
I don't think we can provide an image to the engine; it works with the camera.

On some devices, it seemed that ml kit only works when we store the image on the storage and let ml kit read it from there.

Again, that's supposed to be dealt with the external mlkit package. Anyway there's no related parameter in the package. If you could find the related issue, we should contact the "problematic" users for additional tests.
@teolemon Additional thought: now we can easily create automated screenshots of the home page of the app, if we just create a specific "scanner type" called "screenshot" that displays a saved picture, with the zxing visor on top.

I was thinking on how to make the carousel feel immersive. So far, I only have half-convincing/bad ideas.

get rid of the carousel and loose the compare feature, and move to a card based UI at the bottom of the screen
make the carousel edge to edge, and get rid of the card metaphor
some kind of very low resolution background for the carousel, based on the dominant colors of the camera stream
Find a way to move the focus to the bottom up of the screen
Your idea here

I was thinking on how to make the carousel feel immersive. So far, I only have half-convincing/bad ideas.

I've just created #3777.
In the current PR we change flutter version and 78 files are created/deleted/changed, therefore I suggest to do things step by step, issue by issue, PR by PR.

Thank you @adilwahla @M123-dev @teolemon for your reviews, feedbacks and commits!
It's not that I'm shy but I'm not sure when I can merge this PR, regarding the 4.6.0 version?

Feel free to merge, the release is done


Document how to create and test a package locally

Feedback
An extension author might want to develop his own extension and test it before submitting a PR to the repository for the extension to be considered. Since this process is possible, it should be documented.
Suggested Changes
A doc page should be created detailing the process:

Create the source extension directory
Add the extension.yaml and the extension's descriptor
Bundle your extension
Push the extension into a registry
Add your extension to the extension list at $XDG_DATA_HOME/tanzu-repository/metadata/latest/metadata.yaml
Drop the extension locally in the directory $XDG_DATA_HOME/tanzu-repository/extensions/latest/

Just copy any of the existing extensions on that directory, rename the directory and fix the extension.yaml file and the extensions's bundle registry reference

PS: I'm happy to contribute this in a PR with example snippets for all the steps
Details
TCE build version: v0.1.0

@jorgemoralespou Given the current state of TCE documentation, would you consider this is closed?


Add GL certificate example to RN

Add Greenlight certificate retrieval to the react native example app and readme

Closing this as it would be opinionated


Threshold-based indicators could use more generalised names

There are more than a few indicators that I would like to use with such drastically different thresholds that the meaning of their names is rendered obsolete.
For example, computing tropical_nights with a threshold of 10 degC is hardly "tropical". A more generalized version would be to have tropical_nights with a specific, literature-based threshold value and a tn_days_above (with indicator mapping).
It would be great to have a discussion about this as it would potentially be a breaking change (assuming we remove existing mappings). Perhaps once we can put together a list of potential candidates for generalization.
Generalisable indices (WIP)

[x]  tropical_nights → tn_days_above (#715)
[x] daily_freezthaw_cycles → multiday_temperature_swing (with window=1, op="count") (#717)
[ ] ice_days → tx_days_below
[ ] frost-days → tn_days_below
...

I agree (especially for the case of tropical nights).   Not sure if we should keep both the specific tropical_nights  and the general tn_days_above indicators.
Possibility of building Yaml custom modules can now provide these "common name" indices  for those that prefer this nomenclature etc.  Basically I would lean towards simply renaming tropical_nights to a general tn_days_above  and let users customize based on preference via the yml customization

Note that having dedicated specialized functions has the advantage of allowing us to add custom documentation with references and examples. I agree its code duplication, but it can be useful nonetheless.


Abstract common events

remove unnecessary junk/dead code
LOW_LOWEST -> LOWER
abstract common events

not ready to merge. meant to open as a draft

process_replay is only failing because of the events order. it passes with some local changes that sort carEvents and carState.events
diff --git a/selfdrive/test/process_replay/compare_logs.py b/selfdrive/test/process_replay/compare_logs.py
index 82e701a..0c73914 100755
--- a/selfdrive/test/process_replay/compare_logs.py
+++ b/selfdrive/test/process_replay/compare_logs.py
@@ -47,6 +47,23 @@ def compare_logs(log1, log2, ignore=[]):
       print(msg1, msg2)
       assert False, "msgs not aligned between logs"

+    if msg1.which() == "carState":
+      msg1 = msg1.as_builder()
+      msg1.carState.events = sorted(msg1.carState.events, key=lambda evt: evt.name)
+      msg1 = msg1.as_reader()
+      msg2 = msg2.as_builder()
+      msg2.carState.events = sorted(msg2.carState.events, key=lambda evt: evt.name)
+      msg2 = msg2.as_reader()
+
+
+    if msg1.which() == "carEvents":
+      msg1 = msg1.as_builder()
+      msg1.carEvents = sorted(msg1.carEvents, key=lambda evt: evt.name)
+      msg1 = msg1.as_reader()
+      msg2 = msg2.as_builder()
+      msg2.carEvents = sorted(msg2.carEvents, key=lambda evt: evt.name)
+      msg2 = msg2.as_reader()
+
     msg1_bytes = remove_ignored_fields(msg1, ignore).as_builder().to_bytes()
     msg2_bytes = remove_ignored_fields(msg2, ignore).as_builder().to_bytes()

replay failures:

subaru didn't check for wrongCarMode before. difference is on first frame where the default value is used for cruiseState.available until it gets the msg on can
order of events has changed. lists are ordered in python, so it doesn't make sense to change process_replay to ignore that, so I've been running the test locally using the diff above just for this

I've been running process_replay locally as I go and everything's the same, but VW doesn't have a route yet so it should get some extra scrutiny

@quillford , looks good to me. Can you fix the linter?

I've been running process_replay locally as I go and everything's the same, but VW doesn't have a route yet so it should get some extra scrutiny

What behaviors do you want to see in the way of route data to play under testing? I can supply from my Golf R, I already have to drive a new route for the Panda safety work.

linter should be happy now

@jyoung8607 , just supplied a VW segment myself. @pd0wm will include it in the regression test.
Willem, can you please also regen the test ref? Then this PR is good to go.

@quillford just pushed VW tests to master. Can you rebase? Then I'll generate a new reference file for this PR and we can merge it.

446c1fd961444b88cceecb962ca336aaf92e3ee4 can we used as ref_commit to fix the tests.

Ah, https://github.com/commaai/openpilot/pull/1138 was not merged into your branch. I'll generate a new test ref.

0491cad238025a44b7fc636cf03efc27d80dcdae

  File "/usr/local/pyenv/versions/3.7.3/lib/python3.7/threading.py", line 917, in _bootstrap_inner
    self.run()
  File "/usr/local/pyenv/versions/3.7.3/lib/python3.7/threading.py", line 865, in run
    self._target(*self._args, **self._kwargs)
  File "/home/batman/openpilot/selfdrive/controls/controlsd.py", line 591, in main
    controlsd_thread(sm, pm, logcan)
  File "/home/batman/openpilot/selfdrive/controls/controlsd.py", line 532, in controlsd_thread
    CS, events, cal_perc, mismatch_counter, can_error_counter = data_sample(CI, CC, sm, can_sock, state, mismatch_counter, can_error_counter, params)
  File "/home/batman/openpilot/selfdrive/controls/controlsd.py", line 73, in data_sample
    CS = CI.update(CC, can_strs)
  File "/home/batman/openpilot/selfdrive/car/subaru/interface.py", line 66, in update
    be.type = ButtonType.accelCruise
NameError: name 'ButtonType' is not defined


CT assignment: Not a valid application point

The following is from a comment on issue #769. Opening as a new issue because it seems unrelated to the underlying problem in #769:
I have tried with version 0.68.3 and looks like link can be removed without issues but if I try to add it again after all I got attached error. When I re-run terraform apply it works.

Originally posted by @pyspiotr in https://github.com/Juniper/terraform-provider-apstra/issues/769#issuecomment-2288296961

Hi Chris,
Here are the steps:

Delete two links - output after apply

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
~ update in-place

destroy
<= read (data resources)

Terraform will perform the following actions:
data.apstra_datacenter_interfaces_by_link_tag.DC1_AP_VNET_VDI_SERVER_GROUP_1 will be read during apply
(depends on a resource or a module with changes pending)
<= data "apstra_datacenter_interfaces_by_link_tag" "DC1_AP_VNET_VDI_SERVER_GROUP_1" {
+ blueprint_id = "558b40a1-59ef-4a30-ad4e-9d6c054577ae"
+ graph_query  = (known after apply)
+ ids          = (known after apply)
+ tags         = [
+ "VNET_VDI_SERVER_GROUP_1",
]
}
apstra_datacenter_connectivity_template_assignments.DC1_VNET_VDI_SERVER_GROUP_1 will be updated in-place
~ resource "apstra_datacenter_connectivity_template_assignments" "DC1_VNET_VDI_SERVER_GROUP_1" {
~ application_point_ids    = [
- "jsSYU3UZmtR6HljdGnk",
] -> (known after apply)
+ ip_link_ids              = (known after apply)
# (2 unchanged attributes hidden)
}
apstra_datacenter_connectivity_template_assignments.DC1_VNET_VDI_SERVER_GROUP_1_NFS will be destroyed
(because apstra_datacenter_connectivity_template_assignments.DC1_VNET_VDI_SERVER_GROUP_1_NFS is not in configuration)

resource "apstra_datacenter_connectivity_template_assignments" "DC1_VNET_VDI_SERVER_GROUP_1_NFS" {

application_point_ids    = [

"cgMi-20rUylrfPt7yhg",
] -> null

blueprint_id             = "558b40a1-59ef-4a30-ad4e-9d6c054577ae" -> null
connectivity_template_id = "11949259-638e-11ef-81f0-617073747261" -> null
}

apstra_datacenter_generic_system.DC1_LABS92135 will be updated in-place
~ resource "apstra_datacenter_generic_system" "DC1_LABS92135" {
id                   = "EJubKRuzZmA48mvR-44"
~ links                = [
- {
- group_label                   = "LABS92135_bond1" -> null
- lag_mode                      = "lacp_active" -> null
- tags                          = [
- "10G",
- "SERVER",
- "VNET_VDI_SERVER_GROUP_1_NFS",
] -> null
- target_switch_id              = "Zx82yoWoOYIeqXpQsaU" -> null
- target_switch_if_name         = "xe-0/0/1" -> null
- target_switch_if_transform_id = 2 -> null
},
- {
- group_label                   = "LABS92135_bond1" -> null
- lag_mode                      = "lacp_active" -> null
- tags                          = [
- "10G",
- "SERVER",
- "VNET_VDI_SERVER_GROUP_1_NFS",
] -> null
- target_switch_id              = "ypPvIFe-rU-GS5yRoRY" -> null
- target_switch_if_name         = "xe-0/0/1" -> null
- target_switch_if_transform_id = 2 -> null
},
# (2 unchanged elements hidden)
]
name                 = "LABS92135"
tags                 = [
"SERVER",
]
# (7 unchanged attributes hidden)
}
Plan: 0 to add, 2 to change, 1 to destroy.
Do you want to perform these actions?
Terraform will perform the actions described above.
Only 'yes' will be accepted to approve.
Enter a value: yes
apstra_datacenter_connectivity_template_assignments.DC1_VNET_VDI_SERVER_GROUP_1_NFS: Destroying...
yeapstra_datacenter_connectivity_template_assignments.DC1_VNET_VDI_SERVER_GROUP_1_NFS: Destruction complete after 2s
apstra_datacenter_generic_system.DC1_LABS92135: Modifying... [id=EJubKRuzZmA48mvR-44]
apstra_datacenter_generic_system.DC1_LABS92135: Modifications complete after 1s [id=EJubKRuzZmA48mvR-44]
data.apstra_datacenter_interfaces_by_link_tag.DC1_AP_VNET_VDI_SERVER_GROUP_1: Reading...
data.apstra_datacenter_interfaces_by_link_tag.DC1_AP_VNET_VDI_SERVER_GROUP_1: Read complete after 0s
Apply complete! Resources: 0 added, 1 changed, 1 destroyed.

Re-added links and executed terraform apply

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:

create
~ update in-place
-/+ destroy and then create replacement
<= read (data resources)

Terraform will perform the following actions:
data.apstra_datacenter_interfaces_by_link_tag.DC1_AP_VNET_VDI_SERVER_GROUP_1 will be read during apply
(depends on a resource or a module with changes pending)
<= data "apstra_datacenter_interfaces_by_link_tag" "DC1_AP_VNET_VDI_SERVER_GROUP_1" {
+ blueprint_id = "558b40a1-59ef-4a30-ad4e-9d6c054577ae"
+ graph_query  = (known after apply)
+ ids          = (known after apply)
+ tags         = [
+ "VNET_VDI_SERVER_GROUP_1",
]
}
data.apstra_datacenter_interfaces_by_link_tag.DC1_AP_VNET_VDI_SERVER_GROUP_1_NFS will be read during apply
(depends on a resource or a module with changes pending)
<= data "apstra_datacenter_interfaces_by_link_tag" "DC1_AP_VNET_VDI_SERVER_GROUP_1_NFS" {
+ blueprint_id = "558b40a1-59ef-4a30-ad4e-9d6c054577ae"
+ graph_query  = (known after apply)
+ ids          = (known after apply)
+ tags         = [
+ "VNET_VDI_SERVER_GROUP_1_NFS",
]
}
apstra_datacenter_connectivity_template_assignments.DC1_VNET_VDI_SERVER_GROUP_1 will be updated in-place
~ resource "apstra_datacenter_connectivity_template_assignments" "DC1_VNET_VDI_SERVER_GROUP_1" {
~ application_point_ids    = [
- "jsSYU3UZmtR6HljdGnk",
] -> (known after apply)
+ ip_link_ids              = (known after apply)
# (2 unchanged attributes hidden)
}
apstra_datacenter_connectivity_template_assignments.DC1_VNET_VDI_SERVER_GROUP_1_NFS will be created

resource "apstra_datacenter_connectivity_template_assignments" "DC1_VNET_VDI_SERVER_GROUP_1_NFS" {

application_point_ids    = (known after apply)
blueprint_id             = "558b40a1-59ef-4a30-ad4e-9d6c054577ae"
connectivity_template_id = "11949259-638e-11ef-81f0-617073747261"
ip_link_ids              = (known after apply)
}

apstra_datacenter_generic_system.DC1_LABS92135 must be replaced
-/+ resource "apstra_datacenter_generic_system" "DC1_LABS92135" {
~ id                   = "EJubKRuzZmA48mvR-44" -> (known after apply)
~ links                = [
+ { # forces replacement
+ group_label                   = "LABS92135_bond1"
+ lag_mode                      = "lacp_active"
+ tags                          = [
+ "10G",
+ "SERVER",
+ "VNET_VDI_SERVER_GROUP_1_NFS",
]
+ target_switch_id              = "Zx82yoWoOYIeqXpQsaU"
+ target_switch_if_name         = "xe-0/0/1"
+ target_switch_if_transform_id = 2
},
+ { # forces replacement
+ group_label                   = "LABS92135_bond1"
+ lag_mode                      = "lacp_active"
+ tags                          = [
+ "10G",
+ "SERVER",
+ "VNET_VDI_SERVER_GROUP_1_NFS",
]
+ target_switch_id              = "ypPvIFe-rU-GS5yRoRY"
+ target_switch_if_name         = "xe-0/0/1"
+ target_switch_if_transform_id = 2
},
# (2 unchanged elements hidden)
]
name                 = "LABS92135"
tags                 = [
"SERVER",
]
# (7 unchanged attributes hidden)
}
Plan: 2 to add, 1 to change, 1 to destroy.
Do you want to perform these actions?
Terraform will perform the actions described above.
Only 'yes' will be accepted to approve.
Enter a value: yes
apstra_datacenter_generic_system.DC1_LABS92135: Destroying... [id=EJubKRuzZmA48mvR-44]
apstra_datacenter_generic_system.DC1_LABS92135: Destruction complete after 4s
apstra_datacenter_generic_system.DC1_LABS92135: Creating...
apstra_datacenter_generic_system.DC1_LABS92135: Creation complete after 4s [id=shwSci-s3MrfVrt43_U]
data.apstra_datacenter_interfaces_by_link_tag.DC1_AP_VNET_VDI_SERVER_GROUP_1: Reading...
data.apstra_datacenter_interfaces_by_link_tag.DC1_AP_VNET_VDI_SERVER_GROUP_1_NFS: Reading...
data.apstra_datacenter_interfaces_by_link_tag.DC1_AP_VNET_VDI_SERVER_GROUP_1: Read complete after 0s
data.apstra_datacenter_interfaces_by_link_tag.DC1_AP_VNET_VDI_SERVER_GROUP_1_NFS: Read complete after 0s
apstra_datacenter_connectivity_template_assignments.DC1_VNET_VDI_SERVER_GROUP_1_NFS: Creating...
apstra_datacenter_connectivity_template_assignments.DC1_VNET_VDI_SERVER_GROUP_1: Modifying...
apstra_datacenter_connectivity_template_assignments.DC1_VNET_VDI_SERVER_GROUP_1_NFS: Creation complete after 1s
╷
│ Error: failed while assigning Connectivity Template "11948f39-638e-11ef-81f0-617073747261" to Application Points
│
│   with apstra_datacenter_connectivity_template_assignments.DC1_VNET_VDI_SERVER_GROUP_1,
│   on 03_ct_internal_generic_systems_assignment.tf line 6, in resource "apstra_datacenter_connectivity_template_assignments" "DC1_VNET_VDI_SERVER_GROUP_1":
│    6: resource "apstra_datacenter_connectivity_template_assignments" "DC1_VNET_VDI_SERVER_GROUP_1" {
│
│ {"api_response":null,"config_blueprint_version":0,"errors":{"application_points":{"1":{"policies":{"0":"Not a valid application point"}}}},"error_code":422} - http response '' at
│ 'http://apstra-lab/api/blueprints/558b40a1-59ef-4a30-ad4e-9d6c054577ae/obj-policy-batch-apply?async=full&async=full'
╵

Terraform Apply once agin - without any changes.

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
~ update in-place
Terraform will perform the following actions:
apstra_datacenter_connectivity_template_assignments.DC1_VNET_VDI_SERVER_GROUP_1 will be updated in-place
~ resource "apstra_datacenter_connectivity_template_assignments" "DC1_VNET_VDI_SERVER_GROUP_1" {
~ application_point_ids    = [
+ "FNP-dXYYCPyVXG9FVvY",
]
+ ip_link_ids              = (known after apply)
# (2 unchanged attributes hidden)
}
Plan: 0 to add, 1 to change, 0 to destroy.
Do you want to perform these actions?
Terraform will perform the actions described above.
Only 'yes' will be accepted to approve.
Enter a value: yes
apstra_datacenter_connectivity_template_assignments.DC1_VNET_VDI_SERVER_GROUP_1: Modifying...
apstra_datacenter_connectivity_template_assignments.DC1_VNET_VDI_SERVER_GROUP_1: Modifications complete after 1s
Apply complete! Resources: 0 added, 1 changed, 0 destroyed.

Hi @pyspiotr,
I can't reproduce these results without some example before/after configuration, but I think I might know what's going on here.
If you're deleting links from a generic system in the same project which discovers switch ports and then assigns CTs to those ports, then here's what might be happening:

planning phase plans deletion of server links
planning phase enumerates current switch ports
planning phase plans assignment of CTs to enumerated switch ports
apply phase deletes server links
apply phase cannot assign CT to switch ports which existed during plan, but were deleted in the previous step

There's not much we can do about this general category of problem from the provider side. The same thing would happen, for example, in AWS if you discovered (by data source) EC2 instances, planned EBS volume attachment for each, and also deleted an EC2 instance all in one shot: There would be an EBS volume attachment planned for an instance which is being terminated during that apply. It may or may not surface as an error depending on Terraform's execution order and timing.
One defense against this kind of problem is project structure. We have some users who do physical topology work (systems and links) in one project (separate terraform state file, etc...) and network services (VLANs, BGP sessions, etc...) in another project. Their CI system runs these projects independently, and in an order which ensures this problem never appears.
We're planning a provider update with a new feature which will help: It will make leaf/access switch port IDs (map of application point IDs keyed by group_label) an attribute of the apstra_datacenter_generic_system resource. This will eliminate the need for the data source which returns IDs that exist during plan, but which are eliminated during apply.

It occurs to me that you can validate the "link deleted before planned CT assignment completes" theory:

set up the required pre-conditions
run terraform apply, but do not yet type yes to approve the change
Optional - find link IDs in the graph explorer

Click on the "bug" icon in the "staging" tab
Enter node(type='system', label='<server name from the web ui>') in the query box
Hit Execute
Click on the graph tab
There should be only a single "dot" (graph node) on the screen. Grab it and move it into the corner of the screen.
Click Show full blueprint
Connected to the (now moved) system node/dot, you should find interface nodes. Connected to them are link nodes. Connected to the link nodes are other interface nodes.
So, system -> interface -> link <- interface
Note the IDs of the interface nodes at the other side of the link. These are the Application Points.

Type yes so that Terraform executes the changes.
Expect an error.
Hit Refresh in the Graph Explorer - have some application points vanished? been replaced?
Look in Staging -> Tasks for a failed (red) "Assign Connectivity Templates" task
Confirm that "Detailed Status" matches the error you got from terraform
Have a look at the "Request Data" : data->application_points->id - Does it reference an Application Point which disappeared from the graph?

Hi Chris,
I followed your instruction and here are the results:
Until point 6 all is correct including application points been replaced.
Point 7. In Apstra all is good no errors nothing. I can see errors only on terraform side.
Regards,
Piotr

Interesting. I'd have assumed we'd see a failed task in the log, but didn't actually confirm it. Some categories of failed API request don't get logged. I'm not sure about the rules here.
Did some of the application points disappear from the graph?

I've tested the situation (CT assignment API call referencing an application point which no longer exists) manually, and found the failed API call in the tasks screen.
Task list:

Specific task detail page:

Detailed status containing the error we saw in terraform:

I'm pretty confident that we understand what's happening, but I'm surprised you didn't see the failed task. Busy terraform pushed it to a subsequent page?
Javascript didn't refresh the list at the time you checked for it?

Hi Chris, You are right. The error is there. Looks like I miss looked it as it was not visible in the main page.
Regards, Piotr

Hi @pyspiotr,
Thank you for following up. Keep an eye out for the release which adds switch port IDs as attributes of the apstra_datacenter_generic_system resource.
It won't "fix" the problem you ran into here (really there's no fix for attempting to use an object while also deleting it), but the change will re-structure the workflow by eliminating the data lookup, making it much harder to wind up with this sort of problem.
I'm going to close this issue, but am happy to discuss your use case, challenges etc... You can always open another issue or ask your account team to set up a call.


Gitpodify repository

PR to add Gitpod configuration. Accommodates Java 17 requirement for VSC Language server

Thanks, hope this proves useful in practice :)


Localization of images' content

Several pages in other languages have images with text written in English.
For example, the images for JOSM sections - JOSM Plugins in French
A quick review of this appears that some of this has been done for the German section - http://learnosm.org/de/beginner/start-josm/
Second related question and probably warrants a separate issue; having a consistent naming scheme for the images which are stored at: https://github.com/hotosm/learnosm/tree/gh-pages/images
The German images look like they need to be renamed to the scheme used by other languages:
So, the 2nd image used in chapter 3 of beginning section for Dutch is:
nl_beg_ch3_image02.png
This is a pretty big task and probably isn't a priority at the moment (at least not for us in Haiti at the moment). We can use this for discussion and breaking the steps for this.

Standardizing image naming schema
Taking the screenshots
uploading them to here
making the changes in JOSM.

Closing - see #526 and commit on staging site. - working on other issue now


obsolete chronic encephalitis

mass obsoletion
for @twhetzel - the children terms are being rewired to the grandparents

The command for the Simple mass obsoletion pipeline [1] in the mondo.Makefile [2] uses the ROBOT option remove and in reading over how remove works it sounds like re-wiring classes is the default behavior[3], i.e. "...by default: lineage is maintained, and gaps will be filled where classes have been removed. If you wish to not preserve the hierarchy, include --preserve-structure false."
@matentzn - do you think this is the issue with mass obsoletion and the re-wiring of classes?
[1] https://mondo.readthedocs.io/en/latest/editors-guide/merging-and-obsoleting/#simple-mass-obsoletion-pipeline
[2] https://github.com/monarch-initiative/mondo/blob/master/src/ontology/mondo.Makefile#L502
[3] https://robot.obolibrary.org/remove#preserving-the-structure
[4] Direct command for mass_obsolete2:
$(ROBOT) query -i $(SRC) --use-graphs true --update tmp/mass_obsolete.ru \
		remove -T tmp/mass_obsolete_me.txt --axioms logical convert -f obo --check false -o $(SRC).obo

Fantastic detective hunting @twhetzel. YES thats it!
To fix @nicolevasilevsky's issue then, just make a PR on the makefile and add --preserve-structure false to remove, and assign to me for review.


Removing repositories from cloud

Creating a guide on how to remove repositories from cloud

Made final edits to the doc as per your help @indradhanush. Would appreciate your 👀 on it one more time.

I think we need to ensure that the repo is not added under the list of repositories for any code host connection in the instance. Otherwise just deleting the repo from the database and disk does not prevent the next sync job to reclone it.

Yes, that's correct. There's nothing we can really do about user added code, but we can double check that none of our on site level code hosts add the repo.

@indradhanush @ryanslade
Requesting for your 👀 on this final iteration.
Could I please get a 👍🏽 if the doc is all good for merging? 😃


Factor Weighted Cumulative Return plot for each day/period

Factor Weighted Cumulative Return plot
Only the 1 day forward returns graph is plotted in  'Factor Weighted Long/Short Portfolio Cumulative Return'. I expected to see plots for each day/period  forward return ('days' argument passed to create_factor_tear_sheet) .
I might have a factor that shows alpha only after X or Y days and I might not be interested to see 1 day forward returns plots, so I would pass days = (X, Y) to create_factor_tear_sheet and I would expect to see the plots for X and Y forward returns. At least it would be nice to have this feature as an option if not the default behaviour.

https://github.com/quantopian/alphalens/pull/83